[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:06:31 dist-test-slave-q9h9 krb5kdc[14221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:06:31 dist-test-slave-q9h9 krb5kdc[14221](info): set up 2 sockets
Aug 06 13:06:31 dist-test-slave-q9h9 krb5kdc[14221](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 13:06:32.837263 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:06:32 dist-test-slave-q9h9 krb5kdc[14221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:06:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:06:32.957372 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:41521
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:43087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:41521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:33.062631 14236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:33.062858 14236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:33.062891 14236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:33.064040 14236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:33.064082 14236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:33.064101 14236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:33.064126 14236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:33.064142 14236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:06:33.065995 14236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:43087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:41521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:41521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:33.066431 14236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:33.066804 14236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:33.075879 14246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:33.077015 14247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:33.077667 14250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:33.078189 14236 server_base.cc:1034] running on GCE node
I20260806 13:06:33.078583 14236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:33.079097 14236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:33.080252 14236 hybrid_clock.cc:648] HybridClock initialized: now 1786021593080215 us; error 90 us; skew 500 ppm
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021593, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:33.083132 14236 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:06:33.083748 14236 webserver.cc:460] Webserver started at http://127.13.224.126:33473/ using document root <none> and password file <none>
I20260806 13:06:33.084035 14236 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:33.084086 14236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:33.084233 14236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:33.085667 14236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "604e663f504e4f028ad785111548bbfc"
format_stamp: "Formatted at 2026-08-06 13:06:33 on dist-test-slave-q9h9"
server_key: "bd0ec9c8572409ee0838d6ccbefeaed9"
server_key_iv: "72e80b1c24872cccd065ae3ba63bd063"
server_key_version: "encryptionkey@0"
I20260806 13:06:33.085999 14236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "604e663f504e4f028ad785111548bbfc"
format_stamp: "Formatted at 2026-08-06 13:06:33 on dist-test-slave-q9h9"
server_key: "bd0ec9c8572409ee0838d6ccbefeaed9"
server_key_iv: "72e80b1c24872cccd065ae3ba63bd063"
server_key_version: "encryptionkey@0"
I20260806 13:06:33.087911 14236 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 13:06:33.088982 14262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:33.089222 14236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:06:33.089344 14236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "604e663f504e4f028ad785111548bbfc"
format_stamp: "Formatted at 2026-08-06 13:06:33 on dist-test-slave-q9h9"
server_key: "bd0ec9c8572409ee0838d6ccbefeaed9"
server_key_iv: "72e80b1c24872cccd065ae3ba63bd063"
server_key_version: "encryptionkey@0"
I20260806 13:06:33.089440 14236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:33.102862 14236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:33.105149 14236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:33.105348 14236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:33.112468 14236 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:41521
I20260806 13:06:33.112562 14338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:41521 every 8 connection(s)
I20260806 13:06:33.113135 14236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:33.116008 14339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:33.119689 14339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc: Bootstrap starting.
I20260806 13:06:33.120893 14339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:33.121523 14339 log.cc:824] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:33.122465 14339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc: No bootstrap required, opened a new log
I20260806 13:06:33.123558 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 14236
I20260806 13:06:33.123678 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:33.124480 14209 external_mini_cluster.cc:1428] Setting key 9724e3e27d0e23c42212fce694d484f3
I20260806 13:06:33.125330 14339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "604e663f504e4f028ad785111548bbfc" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41521 } }
I20260806 13:06:33.125550 14339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:33.125634 14339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 604e663f504e4f028ad785111548bbfc, State: Initialized, Role: FOLLOWER
I20260806 13:06:33.125835 14339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "604e663f504e4f028ad785111548bbfc" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41521 } }
I20260806 13:06:33.126173 14339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:33.126230 14339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:33.126281 14339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:33.127884 14339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "604e663f504e4f028ad785111548bbfc" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41521 } }
I20260806 13:06:33.128083 14339 leader_election.cc:302] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 604e663f504e4f028ad785111548bbfc; no voters:
I20260806 13:06:33.128328 14339 leader_election.cc:288] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:33.128358 14345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:33.128531 14345 raft_consensus.cc:686] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [term 1 LEADER]: Becoming Leader. State: Replica: 604e663f504e4f028ad785111548bbfc, State: Running, Role: LEADER
I20260806 13:06:33.128676 14345 consensus_queue.cc:237] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "604e663f504e4f028ad785111548bbfc" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41521 } }
I20260806 13:06:33.128867 14339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:33.129154 14347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 604e663f504e4f028ad785111548bbfc. Latest consensus state: current_term: 1 leader_uuid: "604e663f504e4f028ad785111548bbfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "604e663f504e4f028ad785111548bbfc" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41521 } } }
I20260806 13:06:33.129257 14347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:33.129144 14346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "604e663f504e4f028ad785111548bbfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "604e663f504e4f028ad785111548bbfc" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41521 } } }
I20260806 13:06:33.129343 14346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:33.129556 14349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:33.132287 14349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:33.136380 14349 catalog_manager.cc:1347] Generated new cluster ID: 7730dddc206046448823ad96371ed75e
I20260806 13:06:33.136435 14349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:33.146965 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:33.126221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38798 (local address 127.13.224.126:41521)
0806 13:06:33.126694 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:33.126703 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:33.126896 (+ 193us) server_negotiation.cc:407] Connection header received
0806 13:06:33.129878 (+ 2982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:33.129915 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:33.130053 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:33.130173 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:33.132339 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.133612 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.135509 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.136122 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.136233 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:33.141386 (+ 5153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:33.141390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:33.141390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:33.141402 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:33.143545 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:33.144145 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:33.144151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:33.144151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:33.145321 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:33.145603 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:33.145604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:33.145605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:33.145959 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:33.146170 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:33.146428 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 13:06:33.146636 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":133,"threads_started":1}
I20260806 13:06:33.159139 14349 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:33.159670 14349 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:33.163036 14349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 604e663f504e4f028ad785111548bbfc: Generated new TSK 0
I20260806 13:06:33.163192 14349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:06:33.196624 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:41521
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:43087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:33.335738 14374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:33.336072 14374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:33.336118 14374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:33.338042 14374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:33.338114 14374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:33.338160 14374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:06:33.340866 14374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:43087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:41521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:33.341212 14374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:33.341563 14374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:33.348562 14381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:33.348569 14384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:33.348685 14382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:33.349774 14374 server_base.cc:1034] running on GCE node
I20260806 13:06:33.349994 14374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:33.350310 14374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:33.351498 14374 hybrid_clock.cc:648] HybridClock initialized: now 1786021593351477 us; error 36 us; skew 500 ppm
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021593, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:33.354133 14374 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:06:33.354697 14374 webserver.cc:460] Webserver started at http://127.13.224.65:34019/ using document root <none> and password file <none>
I20260806 13:06:33.354956 14374 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:33.355003 14374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:33.355139 14374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:33.356367 14374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "9bbe2f07e04e4f61941207437f10e6f9"
format_stamp: "Formatted at 2026-08-06 13:06:33 on dist-test-slave-q9h9"
server_key: "87a2ad48a885448dccc704940ba80e32"
server_key_iv: "4b5977047ed3cd6f55441eb11c744722"
server_key_version: "encryptionkey@0"
I20260806 13:06:33.356761 14374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "9bbe2f07e04e4f61941207437f10e6f9"
format_stamp: "Formatted at 2026-08-06 13:06:33 on dist-test-slave-q9h9"
server_key: "87a2ad48a885448dccc704940ba80e32"
server_key_iv: "4b5977047ed3cd6f55441eb11c744722"
server_key_version: "encryptionkey@0"
I20260806 13:06:33.358234 14374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:06:33.359072 14394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:33.359266 14374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:06:33.359361 14374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "9bbe2f07e04e4f61941207437f10e6f9"
format_stamp: "Formatted at 2026-08-06 13:06:33 on dist-test-slave-q9h9"
server_key: "87a2ad48a885448dccc704940ba80e32"
server_key_iv: "4b5977047ed3cd6f55441eb11c744722"
server_key_version: "encryptionkey@0"
I20260806 13:06:33.359437 14374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:33.368786 14374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:33.371739 14374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:33.371913 14374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:33.372287 14374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:33.372747 14374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:33.372874 14374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:33.372924 14374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:33.372956 14374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:33.385637 14374 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:39825
I20260806 13:06:33.385679 14566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:39825 every 8 connection(s)
I20260806 13:06:33.386368 14374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:06:33.388576 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 14374
I20260806 13:06:33.388669 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:06:33.388814 14209 external_mini_cluster.cc:1428] Setting key ad88876282af6ea7e6ed2ebe21822418
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021593, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
I20260806 13:06:33.404635 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:33.387734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:33125 (local address 127.13.224.126:41521)
0806 13:06:33.387785 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:33.387787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:33.387926 (+ 139us) server_negotiation.cc:407] Connection header received
0806 13:06:33.388644 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:33.388646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:33.388662 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:33.388694 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:33.390389 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.391159 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.392873 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.393160 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.393199 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:33.395865 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:33.395867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:33.395868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:33.395875 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:33.398068 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:33.403424 (+ 5356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:33.403426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:33.403426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:33.403499 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:33.403852 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:33.403853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:33.403853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:33.403998 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:33.404055 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:33.404419 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 13:06:33.404515 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 13:06:33.405171 14570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:33.387595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:41521 (local address 127.13.224.65:33125)
0806 13:06:33.387791 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:33.387840 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:33.388503 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:33.388882 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:33.388886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:33.389895 (+ 1009us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:33.390265 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:33.390272 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.391284 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.391285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:33.392778 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:33.392781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.393301 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.393302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:33.393436 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:33.393949 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:33.393959 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:33.395756 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:33.398170 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:33.398178 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:33.398180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:33.398527 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:33.403599 (+ 5072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:33.403600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:33.403601 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:33.403742 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:33.404092 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:33.404098 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:33.404262 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 13:06:33.404744 (+ 482us) client_negotiation.cc:240] Negotiation successful
0806 13:06:33.404939 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
I20260806 13:06:33.406275 14567 heartbeater.cc:347] Connected to a master server at 127.13.224.126:41521
I20260806 13:06:33.406396 14567 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:33.406600 14567 heartbeater.cc:511] Master 127.13.224.126:41521 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
I20260806 13:06:33.407546 14285 ts_manager.cc:194] Registered new tserver with Master: 9bbe2f07e04e4f61941207437f10e6f9 (127.13.224.65:39825)
I20260806 13:06:33.408561 14285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:33125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:06:33.425506 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:41521
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:43087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:33.525125 14576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:33.525321 14576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:33.525345 14576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:33.526531 14576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:33.526584 14576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:33.526612 14576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:06:33.528268 14576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:43087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:41521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:33.528518 14576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:33.528759 14576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:33.533648 14587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:33.533675 14584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:33.533658 14585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:33.533903 14576 server_base.cc:1034] running on GCE node
I20260806 13:06:33.534137 14576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:33.534363 14576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:33.535501 14576 hybrid_clock.cc:648] HybridClock initialized: now 1786021593535456 us; error 61 us; skew 500 ppm
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021593, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:33.539858 14576 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:06:33.540362 14576 webserver.cc:460] Webserver started at http://127.13.224.66:36143/ using document root <none> and password file <none>
I20260806 13:06:33.540576 14576 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:33.540624 14576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:33.540758 14576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:33.541899 14576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "8b3ef5ef5a384eabae174010f7a5686a"
format_stamp: "Formatted at 2026-08-06 13:06:33 on dist-test-slave-q9h9"
server_key: "55779703f8bb4a50d2ffcb0feb0ef6a8"
server_key_iv: "61a976edc52786a0358a37bc2d588df4"
server_key_version: "encryptionkey@0"
I20260806 13:06:33.542309 14576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "8b3ef5ef5a384eabae174010f7a5686a"
format_stamp: "Formatted at 2026-08-06 13:06:33 on dist-test-slave-q9h9"
server_key: "55779703f8bb4a50d2ffcb0feb0ef6a8"
server_key_iv: "61a976edc52786a0358a37bc2d588df4"
server_key_version: "encryptionkey@0"
I20260806 13:06:33.543829 14576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:06:33.544684 14597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:33.544835 14576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:33.544934 14576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "8b3ef5ef5a384eabae174010f7a5686a"
format_stamp: "Formatted at 2026-08-06 13:06:33 on dist-test-slave-q9h9"
server_key: "55779703f8bb4a50d2ffcb0feb0ef6a8"
server_key_iv: "61a976edc52786a0358a37bc2d588df4"
server_key_version: "encryptionkey@0"
I20260806 13:06:33.545007 14576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:33.570812 14576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:33.572700 14576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:33.572814 14576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:33.573014 14576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:33.573282 14576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:33.573312 14576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:33.573333 14576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:33.573350 14576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:33.581921 14576 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:34255
I20260806 13:06:33.581961 14747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:34255 every 8 connection(s)
I20260806 13:06:33.582365 14576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:06:33.584641 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 14576
I20260806 13:06:33.584748 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:06:33.584916 14209 external_mini_cluster.cc:1428] Setting key 7f5dbd29d291607af8d5e125c124dc82
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021593, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:33.592039 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:33.583119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:33393 (local address 127.13.224.126:41521)
0806 13:06:33.583171 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:33.583173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:33.583363 (+ 190us) server_negotiation.cc:407] Connection header received
0806 13:06:33.583871 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:33.583872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:33.583882 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:33.583930 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:33.585038 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.585651 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.586965 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.587152 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.587210 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:33.589270 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:33.589271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:33.589272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:33.589279 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:33.590815 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:33.591177 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:33.591178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:33.591178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:33.591239 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:33.591441 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:33.591442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:33.591442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:33.591570 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:33.591646 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:33.591845 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 13:06:33.591926 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 13:06:33.592175 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:33.583098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:41521 (local address 127.13.224.66:33393)
0806 13:06:33.583251 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:33.583287 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:33.583786 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:33.583985 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:33.583988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:33.584640 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:33.584935 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:33.584943 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.585738 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.585739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:33.586887 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:33.586891 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.587231 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.587233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:33.587350 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:33.587805 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:33.587813 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:33.589175 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:33.590890 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:33.590895 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:33.590897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:33.591109 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:33.591303 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:33.591304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:33.591304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:33.591392 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:33.591625 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:33.591627 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:33.591734 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 13:06:33.591808 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 13:06:33.591918 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260806 13:06:33.592577 14748 heartbeater.cc:347] Connected to a master server at 127.13.224.126:41521
I20260806 13:06:33.592700 14748 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:33.592886 14748 heartbeater.cc:511] Master 127.13.224.126:41521 requested a full tablet report, sending...
I20260806 13:06:33.593300 14285 ts_manager.cc:194] Registered new tserver with Master: 8b3ef5ef5a384eabae174010f7a5686a (127.13.224.66:34255)
I20260806 13:06:33.593681 14285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:33393
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:06:33.608129 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:41521
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:43087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:33.703787 14753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:33.703980 14753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:33.704005 14753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:33.705129 14753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:33.705178 14753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:33.705210 14753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:06:33.706710 14753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:43087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:41521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:33.706918 14753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:33.707129 14753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:33.711814 14759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:33.711805 14763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:33.711805 14758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:33.712054 14753 server_base.cc:1034] running on GCE node
I20260806 13:06:33.712266 14753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:33.712494 14753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:33.713629 14753 hybrid_clock.cc:648] HybridClock initialized: now 1786021593713610 us; error 33 us; skew 500 ppm
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021593, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:33.716233 14753 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:06:33.716689 14753 webserver.cc:460] Webserver started at http://127.13.224.67:44163/ using document root <none> and password file <none>
I20260806 13:06:33.716884 14753 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:33.716936 14753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:33.717057 14753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:33.718063 14753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "36824f6b8ff74f759aa1b7baa9a002b1"
format_stamp: "Formatted at 2026-08-06 13:06:33 on dist-test-slave-q9h9"
server_key: "8f0513c7d6f8032a8862c236bf6fdaac"
server_key_iv: "0e9906c721be6815848386f2995c07f1"
server_key_version: "encryptionkey@0"
I20260806 13:06:33.718408 14753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "36824f6b8ff74f759aa1b7baa9a002b1"
format_stamp: "Formatted at 2026-08-06 13:06:33 on dist-test-slave-q9h9"
server_key: "8f0513c7d6f8032a8862c236bf6fdaac"
server_key_iv: "0e9906c721be6815848386f2995c07f1"
server_key_version: "encryptionkey@0"
I20260806 13:06:33.719908 14753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:06:33.720539 14771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:33.720695 14753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:33.720784 14753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "36824f6b8ff74f759aa1b7baa9a002b1"
format_stamp: "Formatted at 2026-08-06 13:06:33 on dist-test-slave-q9h9"
server_key: "8f0513c7d6f8032a8862c236bf6fdaac"
server_key_iv: "0e9906c721be6815848386f2995c07f1"
server_key_version: "encryptionkey@0"
I20260806 13:06:33.720835 14753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:33.738782 14753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:33.740454 14753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:33.740558 14753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:33.740758 14753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:33.741007 14753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:33.741037 14753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:33.741056 14753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:33.741073 14753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:33.750442 14753 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:37871
I20260806 13:06:33.750486 14923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:37871 every 8 connection(s)
I20260806 13:06:33.750837 14753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:06:33.752579 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 14753
I20260806 13:06:33.752664 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:06:33.752811 14209 external_mini_cluster.cc:1428] Setting key a52f39edfcd22900a248e81c9545f086
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021593, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:33.759359 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:33.751659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:45399 (local address 127.13.224.126:41521)
0806 13:06:33.751715 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:33.751716 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:33.751921 (+ 205us) server_negotiation.cc:407] Connection header received
0806 13:06:33.752415 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:33.752416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:33.752430 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:33.752481 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:33.753424 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.753919 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.754906 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.755090 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.755145 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:33.757057 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:33.757059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:33.757059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:33.757067 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:33.758161 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:33.758518 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:33.758520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:33.758520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:33.758574 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:33.758783 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:33.758784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:33.758785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:33.758889 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:33.758944 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:33.759191 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 13:06:33.759269 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 13:06:33.759663 14927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:33.751636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:41521 (local address 127.13.224.67:45399)
0806 13:06:33.751809 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:33.751844 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:33.752322 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:33.752549 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:33.752553 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:33.753136 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:33.753333 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:33.753339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.754000 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.754001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:33.754826 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:33.754828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.755181 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.755182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:33.755313 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:33.755711 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:33.755718 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:33.756972 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:33.758224 (+ 1252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:33.758232 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:33.758234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:33.758453 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:33.758628 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:33.758629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:33.758629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:33.758727 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:33.758942 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:33.758949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:33.759076 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 13:06:33.759168 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 13:06:33.759331 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260806 13:06:33.760095 14925 heartbeater.cc:347] Connected to a master server at 127.13.224.126:41521
I20260806 13:06:33.760198 14925 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:33.760370 14925 heartbeater.cc:511] Master 127.13.224.126:41521 requested a full tablet report, sending...
I20260806 13:06:33.760713 14285 ts_manager.cc:194] Registered new tserver with Master: 36824f6b8ff74f759aa1b7baa9a002b1 (127.13.224.67:37871)
I20260806 13:06:33.761060 14285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:45399
I20260806 13:06:33.764573 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021593, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021593, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.224.65@KRBTEST.COM
I20260806 13:06:33.787293 14934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:33.780074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33980 (local address 127.13.224.65:39825)
0806 13:06:33.780260 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:33.780264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:33.780274 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:06:33.780298 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:33.780300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:33.780314 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:33.780380 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:33.780882 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.781636 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.782341 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.782927 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.782994 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:33.784910 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:33.784918 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:33.784918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:33.784925 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:33.786409 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:33.786685 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:33.786688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:33.786688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:33.786741 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:33.786883 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:33.786883 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:33.786884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:33.786993 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:33.787108 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:33.787139 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 13:06:33.787218 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
W20260806 13:06:33.787750 14431 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33980
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.65@KRBTEST.COM
I20260806 13:06:33.809211 14934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:33.802654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33990 (local address 127.13.224.65:39825)
0806 13:06:33.802708 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:33.802711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:33.802813 (+ 102us) server_negotiation.cc:407] Connection header received
0806 13:06:33.802871 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:33.802872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:33.802885 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:33.802934 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:33.803449 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.803923 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.804751 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:33.804954 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:33.805012 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:33.807162 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:33.807163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:33.807163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:33.807172 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:33.808233 (+ 1061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:33.808535 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:33.808536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:33.808537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:33.808589 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:33.808759 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:33.808760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:33.808760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:33.808851 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:33.808964 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:33.809044 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 13:06:33.809121 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 13:06:33.809715 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 14374
I20260806 13:06:33.825692 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 14576
I20260806 13:06:33.834232 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 14753
I20260806 13:06:33.842113 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 14236
2026-08-06T13:06:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2356 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14945](info): set up 2 sockets
Aug 06 13:06:33 dist-test-slave-q9h9 krb5kdc[14945](info): commencing operation
krb5kdc: starting...
W20260806 13:06:34.958038 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:06:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:06:35.066627 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:46589
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:46589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:35.155001 14966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:35.155192 14966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:35.155212 14966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:35.156316 14966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:35.156355 14966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:35.156368 14966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:35.156380 14966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:35.156402 14966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:06:35.157853 14966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:46589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:46589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:35.158095 14966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:35.158305 14966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:35.162952 14971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:35.162942 14980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:35.162963 14974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:35.163437 14966 server_base.cc:1034] running on GCE node
I20260806 13:06:35.163662 14966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:35.163919 14966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:35.165055 14966 hybrid_clock.cc:648] HybridClock initialized: now 1786021595165043 us; error 29 us; skew 500 ppm
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:35.167426 14966 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:06:35.167794 14966 webserver.cc:460] Webserver started at http://127.13.224.126:35859/ using document root <none> and password file <none>
I20260806 13:06:35.167960 14966 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:35.167997 14966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:35.168087 14966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:35.169023 14966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "d4e39bb061a44c15886213c33e7edcf4"
format_stamp: "Formatted at 2026-08-06 13:06:35 on dist-test-slave-q9h9"
server_key: "0e563add06d601abd47b3def26e8639f"
server_key_iv: "84c068acedba539eed1801323d202116"
server_key_version: "encryptionkey@0"
I20260806 13:06:35.169305 14966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "d4e39bb061a44c15886213c33e7edcf4"
format_stamp: "Formatted at 2026-08-06 13:06:35 on dist-test-slave-q9h9"
server_key: "0e563add06d601abd47b3def26e8639f"
server_key_iv: "84c068acedba539eed1801323d202116"
server_key_version: "encryptionkey@0"
I20260806 13:06:35.170553 14966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:06:35.171218 14989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:35.171394 14966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:35.171474 14966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "d4e39bb061a44c15886213c33e7edcf4"
format_stamp: "Formatted at 2026-08-06 13:06:35 on dist-test-slave-q9h9"
server_key: "0e563add06d601abd47b3def26e8639f"
server_key_iv: "84c068acedba539eed1801323d202116"
server_key_version: "encryptionkey@0"
I20260806 13:06:35.171540 14966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:35.181067 14966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:35.182860 14966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:35.182964 14966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:35.188607 14966 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:46589
I20260806 13:06:35.188642 15061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:46589 every 8 connection(s)
I20260806 13:06:35.188953 14966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:35.189843 15062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:35.190834 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 14966
I20260806 13:06:35.190910 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:35.191057 14209 external_mini_cluster.cc:1428] Setting key 247c10f72cfc2b81fe5117c50cc249b5
I20260806 13:06:35.192595 15062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4: Bootstrap starting.
I20260806 13:06:35.193753 15062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:35.194372 15062 log.cc:824] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:35.195032 15062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4: No bootstrap required, opened a new log
I20260806 13:06:35.197165 15062 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4e39bb061a44c15886213c33e7edcf4" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46589 } }
I20260806 13:06:35.197294 15062 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:35.197329 15062 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4e39bb061a44c15886213c33e7edcf4, State: Initialized, Role: FOLLOWER
I20260806 13:06:35.197455 15062 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4e39bb061a44c15886213c33e7edcf4" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46589 } }
I20260806 13:06:35.197530 15062 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:35.197561 15062 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:35.197607 15062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:35.198266 15062 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4e39bb061a44c15886213c33e7edcf4" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46589 } }
I20260806 13:06:35.198377 15062 leader_election.cc:302] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4e39bb061a44c15886213c33e7edcf4; no voters:
I20260806 13:06:35.198540 15062 leader_election.cc:288] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:35.198586 15067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:35.198760 15067 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [term 1 LEADER]: Becoming Leader. State: Replica: d4e39bb061a44c15886213c33e7edcf4, State: Running, Role: LEADER
I20260806 13:06:35.198843 15062 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:35.198905 15067 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4e39bb061a44c15886213c33e7edcf4" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46589 } }
I20260806 13:06:35.199214 15067 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4e39bb061a44c15886213c33e7edcf4. Latest consensus state: current_term: 1 leader_uuid: "d4e39bb061a44c15886213c33e7edcf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4e39bb061a44c15886213c33e7edcf4" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46589 } } }
I20260806 13:06:35.199440 15067 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:35.199376 15068 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4e39bb061a44c15886213c33e7edcf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4e39bb061a44c15886213c33e7edcf4" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46589 } } }
I20260806 13:06:35.200083 15068 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:35.200567 15082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:35.201180 15082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:35.201237 15065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.191518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38778 (local address 127.13.224.126:46589)
0806 13:06:35.191704 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:35.191713 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:35.191745 (+ 32us) server_negotiation.cc:407] Connection header received
0806 13:06:35.192288 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:35.192297 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:35.192358 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:35.192500 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:35.193084 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.194453 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.195193 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.195589 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.195683 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.197824 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:35.197828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:35.197829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:35.197840 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:35.199700 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.200057 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.200061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.200061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.200164 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.200386 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.200387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.200388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.200673 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:35.200809 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:35.200921 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 13:06:35.201046 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260806 13:06:35.202749 15082 catalog_manager.cc:1347] Generated new cluster ID: 1f62430df5574ebbaeee91c8c56d0685
I20260806 13:06:35.202798 15082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:06:35.233721 15082 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:35.234200 15082 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:35.239276 15082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4e39bb061a44c15886213c33e7edcf4: Generated new TSK 0
I20260806 13:06:35.239423 15082 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:06:35.270488 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:46589
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:35.352581 15096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:35.352767 15096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:35.352789 15096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:35.353904 15096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:35.353942 15096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:35.353971 15096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:06:35.355511 15096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:46589
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:35.355732 15096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:35.355954 15096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 13:06:35.360973 15096 server_base.cc:1034] running on GCE node
W20260806 13:06:35.360940 15103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:35.360940 15105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:35.360922 15102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:35.361385 15096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:35.361681 15096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:35.362831 15096 hybrid_clock.cc:648] HybridClock initialized: now 1786021595362814 us; error 31 us; skew 500 ppm
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:35.365200 15096 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:06:35.365697 15096 webserver.cc:460] Webserver started at http://127.13.224.65:45931/ using document root <none> and password file <none>
I20260806 13:06:35.365929 15096 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:35.365981 15096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:35.366144 15096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:35.367110 15096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "7b7f5e339d7a43588e7f2cd002b686f6"
format_stamp: "Formatted at 2026-08-06 13:06:35 on dist-test-slave-q9h9"
server_key: "946faa2f0e20fa1f62608095d04431db"
server_key_iv: "5f970f84014237e5c00a26b559636932"
server_key_version: "encryptionkey@0"
I20260806 13:06:35.367486 15096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "7b7f5e339d7a43588e7f2cd002b686f6"
format_stamp: "Formatted at 2026-08-06 13:06:35 on dist-test-slave-q9h9"
server_key: "946faa2f0e20fa1f62608095d04431db"
server_key_iv: "5f970f84014237e5c00a26b559636932"
server_key_version: "encryptionkey@0"
I20260806 13:06:35.368835 15096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:06:35.369496 15113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:35.369664 15096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:35.369752 15096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "7b7f5e339d7a43588e7f2cd002b686f6"
format_stamp: "Formatted at 2026-08-06 13:06:35 on dist-test-slave-q9h9"
server_key: "946faa2f0e20fa1f62608095d04431db"
server_key_iv: "5f970f84014237e5c00a26b559636932"
server_key_version: "encryptionkey@0"
I20260806 13:06:35.369820 15096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:35.381500 15096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:35.383229 15096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:35.383371 15096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:35.383584 15096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:35.383881 15096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:35.383919 15096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:35.383949 15096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:35.383991 15096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:35.390162 15096 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:46229
I20260806 13:06:35.390196 15278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:46229 every 8 connection(s)
I20260806 13:06:35.390476 15096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:06:35.394894 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 15096
I20260806 13:06:35.394985 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:06:35.395149 14209 external_mini_cluster.cc:1428] Setting key be458005240ad035484aaabffa6e1bf1
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:35.399334 15065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.391169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:42619 (local address 127.13.224.126:46589)
0806 13:06:35.391255 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:35.391258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:35.391413 (+ 155us) server_negotiation.cc:407] Connection header received
0806 13:06:35.392276 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:35.392277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:35.392288 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:35.392340 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:35.392867 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.393357 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.394342 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.394554 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.394616 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.396770 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:35.396772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:35.396772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:35.396779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:35.397992 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.398420 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.398421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.398421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.398482 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.398736 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.398737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.398737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.398867 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:35.398930 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:35.399156 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 13:06:35.399248 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 13:06:35.399401 15281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.391124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:46589 (local address 127.13.224.65:42619)
0806 13:06:35.391300 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:35.391333 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:35.392188 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:35.392397 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:35.392400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:35.392624 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:35.392797 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.392801 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.393425 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.393425 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.394236 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.394238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.394624 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.394624 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.394726 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.395233 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:35.395244 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:35.396671 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:35.398127 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:35.398134 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:35.398136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:35.398344 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:35.398538 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:35.398544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:35.398544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:35.398677 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:35.398930 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:35.398933 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:35.399035 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 13:06:35.399114 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 13:06:35.399223 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
I20260806 13:06:35.399856 15279 heartbeater.cc:347] Connected to a master server at 127.13.224.126:46589
I20260806 13:06:35.399950 15279 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:35.400107 15279 heartbeater.cc:511] Master 127.13.224.126:46589 requested a full tablet report, sending...
I20260806 13:06:35.400642 15017 ts_manager.cc:194] Registered new tserver with Master: 7b7f5e339d7a43588e7f2cd002b686f6 (127.13.224.65:46229)
I20260806 13:06:35.401227 15017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:42619
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:06:35.418579 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:46589
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:35.497259 15285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:35.497435 15285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:35.497466 15285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:35.498616 15285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:35.498656 15285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:35.498698 15285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:06:35.500682 15285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:46589
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:35.500939 15285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:35.501148 15285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:35.505877 15292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:35.505906 15293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:35.505914 15295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:35.505991 15285 server_base.cc:1034] running on GCE node
I20260806 13:06:35.506297 15285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:35.506495 15285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:35.507628 15285 hybrid_clock.cc:648] HybridClock initialized: now 1786021595507610 us; error 38 us; skew 500 ppm
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:35.509974 15285 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:06:35.510454 15285 webserver.cc:460] Webserver started at http://127.13.224.66:44439/ using document root <none> and password file <none>
I20260806 13:06:35.510674 15285 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:35.510721 15285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:35.510855 15285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:35.511781 15285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "123c6836846745c1b3187e4d237dbfcf"
format_stamp: "Formatted at 2026-08-06 13:06:35 on dist-test-slave-q9h9"
server_key: "22226d4d6fa2700d77984ea90a9529d7"
server_key_iv: "6d286b7b48570d88ca89004bb778a746"
server_key_version: "encryptionkey@0"
I20260806 13:06:35.512166 15285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "123c6836846745c1b3187e4d237dbfcf"
format_stamp: "Formatted at 2026-08-06 13:06:35 on dist-test-slave-q9h9"
server_key: "22226d4d6fa2700d77984ea90a9529d7"
server_key_iv: "6d286b7b48570d88ca89004bb778a746"
server_key_version: "encryptionkey@0"
I20260806 13:06:35.513350 15285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 13:06:35.513957 15304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:35.514120 15285 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:35.514200 15285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "123c6836846745c1b3187e4d237dbfcf"
format_stamp: "Formatted at 2026-08-06 13:06:35 on dist-test-slave-q9h9"
server_key: "22226d4d6fa2700d77984ea90a9529d7"
server_key_iv: "6d286b7b48570d88ca89004bb778a746"
server_key_version: "encryptionkey@0"
I20260806 13:06:35.514263 15285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:35.528988 15285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:35.530570 15285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:35.530694 15285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:35.530884 15285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:35.531145 15285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:35.531178 15285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:35.531203 15285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:35.531219 15285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:35.536592 15285 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:34097
I20260806 13:06:35.536657 15474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:34097 every 8 connection(s)
I20260806 13:06:35.536906 15285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:35.542491 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 15285
I20260806 13:06:35.542559 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:06:35.542696 14209 external_mini_cluster.cc:1428] Setting key 0808476745885a275db2648320bf03fd
I20260806 13:06:35.545449 15065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.537579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:48827 (local address 127.13.224.126:46589)
0806 13:06:35.537643 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:35.537645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:35.537820 (+ 175us) server_negotiation.cc:407] Connection header received
0806 13:06:35.538670 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:35.538671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:35.538682 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:35.538731 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:35.539252 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.539723 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.540614 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.540794 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.540836 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.542825 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:35.542827 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:35.542827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:35.542834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:35.544139 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.544510 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.544511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.544511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.544562 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.544833 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.544835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.544835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.544981 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:35.545039 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:35.545323 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 13:06:35.545386 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 13:06:35.545656 15479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.537533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:46589 (local address 127.13.224.66:48827)
0806 13:06:35.537690 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:35.537722 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:35.538580 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:35.538789 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:35.538791 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:35.539015 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:35.539182 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.539186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.539788 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.539789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.540552 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.540554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.540849 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.540850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.540937 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.541364 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:35.541371 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:35.542736 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:35.544223 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:35.544230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:35.544232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:35.544437 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:35.544623 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:35.544623 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:35.544624 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:35.544740 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:35.545048 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:35.545054 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:35.545201 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 13:06:35.545306 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 13:06:35.545456 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260806 13:06:35.546100 15475 heartbeater.cc:347] Connected to a master server at 127.13.224.126:46589
I20260806 13:06:35.546190 15475 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:35.546346 15475 heartbeater.cc:511] Master 127.13.224.126:46589 requested a full tablet report, sending...
I20260806 13:06:35.546705 15017 ts_manager.cc:194] Registered new tserver with Master: 123c6836846745c1b3187e4d237dbfcf (127.13.224.66:34097)
I20260806 13:06:35.547065 15017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:48827
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:06:35.565567 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:46589
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:35.645553 15483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:35.645735 15483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:35.645766 15483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:35.646939 15483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:35.646983 15483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:35.647017 15483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:06:35.648495 15483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:46589
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:35.648723 15483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:35.648923 15483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:35.653628 15491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:35.653638 15494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:35.653644 15492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:35.653890 15483 server_base.cc:1034] running on GCE node
I20260806 13:06:35.654135 15483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:35.654369 15483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:35.655527 15483 hybrid_clock.cc:648] HybridClock initialized: now 1786021595655503 us; error 34 us; skew 500 ppm
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:35.657963 15483 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:06:35.658404 15483 webserver.cc:460] Webserver started at http://127.13.224.67:33143/ using document root <none> and password file <none>
I20260806 13:06:35.658600 15483 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:35.658643 15483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:35.658740 15483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:35.659695 15483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "fbd1c4dbde7943349352a49088c510c3"
format_stamp: "Formatted at 2026-08-06 13:06:35 on dist-test-slave-q9h9"
server_key: "1f1ec7cf4464372ea61d6f3c1cf62ada"
server_key_iv: "0091b4f793915275746315b45a5f07c4"
server_key_version: "encryptionkey@0"
I20260806 13:06:35.659987 15483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "fbd1c4dbde7943349352a49088c510c3"
format_stamp: "Formatted at 2026-08-06 13:06:35 on dist-test-slave-q9h9"
server_key: "1f1ec7cf4464372ea61d6f3c1cf62ada"
server_key_iv: "0091b4f793915275746315b45a5f07c4"
server_key_version: "encryptionkey@0"
I20260806 13:06:35.661196 15483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:06:35.661870 15502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:35.662048 15483 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:35.662129 15483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "fbd1c4dbde7943349352a49088c510c3"
format_stamp: "Formatted at 2026-08-06 13:06:35 on dist-test-slave-q9h9"
server_key: "1f1ec7cf4464372ea61d6f3c1cf62ada"
server_key_iv: "0091b4f793915275746315b45a5f07c4"
server_key_version: "encryptionkey@0"
I20260806 13:06:35.662199 15483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:35.686899 15483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:35.688985 15483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:35.689101 15483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:35.689322 15483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:35.689623 15483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:35.689668 15483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:35.689692 15483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:35.689715 15483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:35.695495 15483 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:34655
I20260806 13:06:35.695566 15662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:34655 every 8 connection(s)
I20260806 13:06:35.695837 15483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:06:35.699813 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 15483
I20260806 13:06:35.699916 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:06:35.700081 14209 external_mini_cluster.cc:1428] Setting key 3534ede56e4e1d048c37451636dc00f0
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:35.704480 15065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.696630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:57707 (local address 127.13.224.126:46589)
0806 13:06:35.696714 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:35.696716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:35.696871 (+ 155us) server_negotiation.cc:407] Connection header received
0806 13:06:35.697722 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:35.697723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:35.697732 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:35.697781 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:35.698416 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.698956 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.699952 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.700235 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.700310 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.702379 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:35.702380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:35.702381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:35.702387 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:35.703480 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.703817 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.703817 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.703818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.703862 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.704037 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.704037 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.704037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.704123 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:35.704167 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:35.704363 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 13:06:35.704417 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 13:06:35.704677 15666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.696599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:46589 (local address 127.13.224.67:57707)
0806 13:06:35.696756 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:35.696791 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:35.697641 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:35.697835 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:35.697838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:35.698121 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:35.698331 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.698335 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.699028 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.699029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.699846 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.699849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.700327 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.700328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.700483 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.700993 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:35.701001 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:35.702300 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:35.703544 (+ 1244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:35.703551 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:35.703553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:35.703759 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:35.703909 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:35.703909 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:35.703910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:35.703994 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:35.704167 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:35.704170 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:35.704269 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 13:06:35.704343 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 13:06:35.704455 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260806 13:06:35.705114 15663 heartbeater.cc:347] Connected to a master server at 127.13.224.126:46589
I20260806 13:06:35.705196 15663 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:35.705330 15663 heartbeater.cc:511] Master 127.13.224.126:46589 requested a full tablet report, sending...
I20260806 13:06:35.705657 15017 ts_manager.cc:194] Registered new tserver with Master: fbd1c4dbde7943349352a49088c510c3 (127.13.224.67:34655)
I20260806 13:06:35.705976 15017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:57707
I20260806 13:06:35.711750 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:06:35.718737 15065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.713466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38784 (local address 127.13.224.126:46589)
0806 13:06:35.713501 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:35.713503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:35.713626 (+ 123us) server_negotiation.cc:407] Connection header received
0806 13:06:35.713678 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:35.713679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:35.713688 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:35.713728 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:35.714298 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.714739 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.715449 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.715634 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.715686 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.716674 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:35.716675 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:35.716676 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:35.716682 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:35.717691 (+ 1009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.717998 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.717999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.717999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.718075 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.718246 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.718247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.718247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.718377 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:35.718460 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:35.718592 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 13:06:35.718657 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:06:35.721050 15017 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:06:35.722949 15017 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:06:35.729387 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.725364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:46229 (local address 127.0.0.1:44112)
0806 13:06:35.726010 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:35.726052 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:35.726102 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:35.726309 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:35.726310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:35.726312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:35.726351 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.726352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.727564 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.727565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.728307 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.728309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.729103 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.729104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.729188 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.729203 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 13:06:35.729262 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 13:06:35.729315 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":54,"threads_started":1}
I20260806 13:06:35.729398 15683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.725826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44112 (local address 127.13.224.65:46229)
0806 13:06:35.725995 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:35.725999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:35.726167 (+ 168us) server_negotiation.cc:407] Connection header received
0806 13:06:35.726210 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:35.726212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:35.726226 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:35.726300 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:35.726416 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.727466 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.728396 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.729019 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.729081 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.729146 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:35.729289 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 13:06:35.729326 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260806 13:06:35.729650 15675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.725359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:34655 (local address 127.0.0.1:52420)
0806 13:06:35.725595 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:35.725629 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:35.725688 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:35.725944 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:35.725947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:35.725949 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:35.726010 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.726032 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.727324 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.727325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.728467 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.728470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.729420 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.729421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.729482 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.729487 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:06:35.729543 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 13:06:35.729582 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":4,"thread_start_us":55,"threads_started":1}
I20260806 13:06:35.729673 15678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.725428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52420 (local address 127.13.224.67:34655)
0806 13:06:35.725608 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:35.725613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:35.725681 (+ 68us) server_negotiation.cc:407] Connection header received
0806 13:06:35.725754 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:35.725756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:35.725771 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:35.725851 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:35.726113 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.727221 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.728570 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.729264 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.729330 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.729384 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:35.729561 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 13:06:35.729605 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260806 13:06:35.730094 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.725360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:34097 (local address 127.0.0.1:50728)
0806 13:06:35.725777 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:35.725786 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:35.725824 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:35.726063 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:35.726064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:35.726067 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:35.726116 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.726118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.727186 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.727187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.728486 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.728488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.729879 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.729880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.729924 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.729927 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:06:35.729976 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 13:06:35.730032 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":29,"thread_start_us":48,"threads_started":1}
I20260806 13:06:35.730156 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.725591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50728 (local address 127.13.224.66:34097)
0806 13:06:35.725813 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:35.725817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:35.725828 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:06:35.725893 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:35.725895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:35.725910 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:35.725980 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:35.726193 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.727094 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.728653 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.729332 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.729385 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.729676 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:35.730037 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 13:06:35.730082 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":110,"threads_started":1}
I20260806 13:06:35.730150 15187 tablet_service.cc:1467] Processing CreateTablet for tablet 2e407e299d034a8983a0b922c7d4cf8c (DEFAULT_TABLE table=test-table [id=f64851132924441ca2eb91798780768e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:35.730512 15187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e407e299d034a8983a0b922c7d4cf8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:35.730688 15565 tablet_service.cc:1467] Processing CreateTablet for tablet 2e407e299d034a8983a0b922c7d4cf8c (DEFAULT_TABLE table=test-table [id=f64851132924441ca2eb91798780768e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:35.731003 15565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e407e299d034a8983a0b922c7d4cf8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:35.730981 15377 tablet_service.cc:1467] Processing CreateTablet for tablet 2e407e299d034a8983a0b922c7d4cf8c (DEFAULT_TABLE table=test-table [id=f64851132924441ca2eb91798780768e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:35.731279 15377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e407e299d034a8983a0b922c7d4cf8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:35.732968 15684 tablet_bootstrap.cc:492] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6: Bootstrap starting.
I20260806 13:06:35.733589 15684 tablet_bootstrap.cc:654] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:35.734131 15685 tablet_bootstrap.cc:492] T 2e407e299d034a8983a0b922c7d4cf8c P fbd1c4dbde7943349352a49088c510c3: Bootstrap starting.
I20260806 13:06:35.734184 15684 log.cc:824] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:35.734480 15686 tablet_bootstrap.cc:492] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf: Bootstrap starting.
I20260806 13:06:35.734717 15685 tablet_bootstrap.cc:654] T 2e407e299d034a8983a0b922c7d4cf8c P fbd1c4dbde7943349352a49088c510c3: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:35.734966 15684 tablet_bootstrap.cc:492] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6: No bootstrap required, opened a new log
I20260806 13:06:35.735038 15684 ts_tablet_manager.cc:1397] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:06:35.735098 15686 tablet_bootstrap.cc:654] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:35.735229 15685 log.cc:824] T 2e407e299d034a8983a0b922c7d4cf8c P fbd1c4dbde7943349352a49088c510c3: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:35.735670 15686 log.cc:824] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:35.736168 15685 tablet_bootstrap.cc:492] T 2e407e299d034a8983a0b922c7d4cf8c P fbd1c4dbde7943349352a49088c510c3: No bootstrap required, opened a new log
I20260806 13:06:35.736228 15685 ts_tablet_manager.cc:1397] T 2e407e299d034a8983a0b922c7d4cf8c P fbd1c4dbde7943349352a49088c510c3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 13:06:35.736420 15686 tablet_bootstrap.cc:492] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf: No bootstrap required, opened a new log
I20260806 13:06:35.736487 15686 ts_tablet_manager.cc:1397] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:06:35.736930 15684 raft_consensus.cc:348] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7f5e339d7a43588e7f2cd002b686f6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 46229 } } peers { permanent_uuid: "fbd1c4dbde7943349352a49088c510c3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34655 } } peers { permanent_uuid: "123c6836846745c1b3187e4d237dbfcf" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 34097 } }
I20260806 13:06:35.737102 15684 raft_consensus.cc:374] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:35.737139 15684 raft_consensus.cc:729] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b7f5e339d7a43588e7f2cd002b686f6, State: Initialized, Role: FOLLOWER
I20260806 13:06:35.737257 15684 consensus_queue.cc:260] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7f5e339d7a43588e7f2cd002b686f6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 46229 } } peers { permanent_uuid: "fbd1c4dbde7943349352a49088c510c3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34655 } } peers { permanent_uuid: "123c6836846745c1b3187e4d237dbfcf" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 34097 } }
I20260806 13:06:35.737524 15684 ts_tablet_manager.cc:1428] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 13:06:35.737701 15279 heartbeater.cc:503] Master 127.13.224.126:46589 was elected leader, sending a full tablet report...
I20260806 13:06:35.737738 15685 raft_consensus.cc:348] T 2e407e299d034a8983a0b922c7d4cf8c P fbd1c4dbde7943349352a49088c510c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7f5e339d7a43588e7f2cd002b686f6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 46229 } } peers { permanent_uuid: "fbd1c4dbde7943349352a49088c510c3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34655 } } peers { permanent_uuid: "123c6836846745c1b3187e4d237dbfcf" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 34097 } }
I20260806 13:06:35.737887 15685 raft_consensus.cc:374] T 2e407e299d034a8983a0b922c7d4cf8c P fbd1c4dbde7943349352a49088c510c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:35.737924 15685 raft_consensus.cc:729] T 2e407e299d034a8983a0b922c7d4cf8c P fbd1c4dbde7943349352a49088c510c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd1c4dbde7943349352a49088c510c3, State: Initialized, Role: FOLLOWER
I20260806 13:06:35.738054 15685 consensus_queue.cc:260] T 2e407e299d034a8983a0b922c7d4cf8c P fbd1c4dbde7943349352a49088c510c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7f5e339d7a43588e7f2cd002b686f6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 46229 } } peers { permanent_uuid: "fbd1c4dbde7943349352a49088c510c3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34655 } } peers { permanent_uuid: "123c6836846745c1b3187e4d237dbfcf" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 34097 } }
I20260806 13:06:35.738390 15663 heartbeater.cc:503] Master 127.13.224.126:46589 was elected leader, sending a full tablet report...
I20260806 13:06:35.738305 15686 raft_consensus.cc:348] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7f5e339d7a43588e7f2cd002b686f6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 46229 } } peers { permanent_uuid: "fbd1c4dbde7943349352a49088c510c3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34655 } } peers { permanent_uuid: "123c6836846745c1b3187e4d237dbfcf" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 34097 } }
I20260806 13:06:35.738482 15686 raft_consensus.cc:374] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:35.738533 15686 raft_consensus.cc:729] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 123c6836846745c1b3187e4d237dbfcf, State: Initialized, Role: FOLLOWER
I20260806 13:06:35.738615 15685 ts_tablet_manager.cc:1428] T 2e407e299d034a8983a0b922c7d4cf8c P fbd1c4dbde7943349352a49088c510c3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:06:35.738646 15686 consensus_queue.cc:260] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7f5e339d7a43588e7f2cd002b686f6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 46229 } } peers { permanent_uuid: "fbd1c4dbde7943349352a49088c510c3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34655 } } peers { permanent_uuid: "123c6836846745c1b3187e4d237dbfcf" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 34097 } }
I20260806 13:06:35.738910 15686 ts_tablet_manager.cc:1428] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 13:06:35.738921 15475 heartbeater.cc:503] Master 127.13.224.126:46589 was elected leader, sending a full tablet report...
W20260806 13:06:35.787424 15478 tablet.cc:2369] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:06:35.818188 15694 raft_consensus.cc:482] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:06:35.818331 15694 raft_consensus.cc:504] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7f5e339d7a43588e7f2cd002b686f6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 46229 } } peers { permanent_uuid: "fbd1c4dbde7943349352a49088c510c3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34655 } } peers { permanent_uuid: "123c6836846745c1b3187e4d237dbfcf" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 34097 } }
I20260806 13:06:35.818701 15694 leader_election.cc:288] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b7f5e339d7a43588e7f2cd002b686f6 (127.13.224.65:46229), fbd1c4dbde7943349352a49088c510c3 (127.13.224.67:34655)
I20260806 13:06:35.821394 15479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.818868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:46229 (local address 127.13.224.66:40521)
0806 13:06:35.818910 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:35.818921 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:35.818962 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:35.819107 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:35.819109 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:35.819111 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:35.819189 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.819191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.819849 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.819850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.820584 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.820586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.821146 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.821147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.821215 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.821218 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:06:35.821272 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 13:06:35.821311 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 13:06:35.821394 15683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.818915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:40521 (local address 127.13.224.65:46229)
0806 13:06:35.818968 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:35.818970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:35.818980 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:06:35.819007 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:35.819008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:35.819017 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:35.819068 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:35.819282 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.819765 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.820682 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.821063 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.821135 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.821145 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:35.821286 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 13:06:35.821325 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 13:06:35.821521 15678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.818941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:57587 (local address 127.13.224.67:34655)
0806 13:06:35.818988 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:35.818990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:35.819233 (+ 243us) server_negotiation.cc:407] Connection header received
0806 13:06:35.819275 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:35.819277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:35.819285 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:35.819326 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:35.819473 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.819922 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.820819 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.821187 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.821253 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.821261 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:35.821427 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 13:06:35.821463 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:06:35.821521 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.818870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:34655 (local address 127.13.224.66:57587)
0806 13:06:35.819174 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:35.819183 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:35.819233 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:35.819336 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:35.819337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:35.819338 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:35.819405 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.819406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.820004 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.820005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.820720 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:35.820721 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.821269 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.821271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:35.821355 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.821359 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:06:35.821403 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 13:06:35.821443 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":135,"thread_start_us":123,"threads_started":1}
I20260806 13:06:35.821852 15213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e407e299d034a8983a0b922c7d4cf8c" candidate_uuid: "123c6836846745c1b3187e4d237dbfcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b7f5e339d7a43588e7f2cd002b686f6" is_pre_election: true
I20260806 13:06:35.822043 15213 raft_consensus.cc:2454] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 123c6836846745c1b3187e4d237dbfcf in term 0.
I20260806 13:06:35.822289 15310 leader_election.cc:302] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 123c6836846745c1b3187e4d237dbfcf, 7b7f5e339d7a43588e7f2cd002b686f6; no voters:
I20260806 13:06:35.822429 15694 raft_consensus.cc:2781] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:06:35.822492 15694 raft_consensus.cc:482] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:06:35.822512 15694 raft_consensus.cc:3037] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:35.823307 15694 raft_consensus.cc:504] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7f5e339d7a43588e7f2cd002b686f6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 46229 } } peers { permanent_uuid: "fbd1c4dbde7943349352a49088c510c3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34655 } } peers { permanent_uuid: "123c6836846745c1b3187e4d237dbfcf" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 34097 } }
I20260806 13:06:35.823565 15694 leader_election.cc:288] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [CANDIDATE]: Term 1 election: Requested vote from peers 7b7f5e339d7a43588e7f2cd002b686f6 (127.13.224.65:46229), fbd1c4dbde7943349352a49088c510c3 (127.13.224.67:34655)
I20260806 13:06:35.823589 15213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e407e299d034a8983a0b922c7d4cf8c" candidate_uuid: "123c6836846745c1b3187e4d237dbfcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b7f5e339d7a43588e7f2cd002b686f6"
I20260806 13:06:35.823645 15213 raft_consensus.cc:3037] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:35.823609 15594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e407e299d034a8983a0b922c7d4cf8c" candidate_uuid: "123c6836846745c1b3187e4d237dbfcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd1c4dbde7943349352a49088c510c3"
I20260806 13:06:35.823724 15594 raft_consensus.cc:3037] T 2e407e299d034a8983a0b922c7d4cf8c P fbd1c4dbde7943349352a49088c510c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:35.821848 15595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e407e299d034a8983a0b922c7d4cf8c" candidate_uuid: "123c6836846745c1b3187e4d237dbfcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd1c4dbde7943349352a49088c510c3" is_pre_election: true
I20260806 13:06:35.824294 15213 raft_consensus.cc:2454] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 123c6836846745c1b3187e4d237dbfcf in term 1.
I20260806 13:06:35.824519 15310 leader_election.cc:302] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 123c6836846745c1b3187e4d237dbfcf, 7b7f5e339d7a43588e7f2cd002b686f6; no voters:
I20260806 13:06:35.824506 15594 raft_consensus.cc:2454] T 2e407e299d034a8983a0b922c7d4cf8c P fbd1c4dbde7943349352a49088c510c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 123c6836846745c1b3187e4d237dbfcf in term 1.
I20260806 13:06:35.824606 15694 raft_consensus.cc:2781] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:35.824759 15694 raft_consensus.cc:686] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [term 1 LEADER]: Becoming Leader. State: Replica: 123c6836846745c1b3187e4d237dbfcf, State: Running, Role: LEADER
I20260806 13:06:35.824842 15694 consensus_queue.cc:237] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7f5e339d7a43588e7f2cd002b686f6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 46229 } } peers { permanent_uuid: "fbd1c4dbde7943349352a49088c510c3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34655 } } peers { permanent_uuid: "123c6836846745c1b3187e4d237dbfcf" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 34097 } }
I20260806 13:06:35.825739 15016 catalog_manager.cc:5495] T 2e407e299d034a8983a0b922c7d4cf8c P 123c6836846745c1b3187e4d237dbfcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 123c6836846745c1b3187e4d237dbfcf (127.13.224.66). New cstate: current_term: 1 leader_uuid: "123c6836846745c1b3187e4d237dbfcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b7f5e339d7a43588e7f2cd002b686f6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 46229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbd1c4dbde7943349352a49088c510c3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "123c6836846745c1b3187e4d237dbfcf" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 34097 } health_report { overall_health: HEALTHY } } }
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.65@KRBTEST.COM
I20260806 13:06:35.848524 15683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.842164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44126 (local address 127.13.224.65:46229)
0806 13:06:35.842228 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:35.842229 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:35.842258 (+ 29us) server_negotiation.cc:407] Connection header received
0806 13:06:35.842315 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:35.842316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:35.842323 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:35.842358 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:35.842850 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.843276 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.843934 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.844099 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.844140 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.846230 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:35.846238 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:35.846238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:35.846252 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:35.847686 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.847958 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.847960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.847961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.848009 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.848143 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.848143 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.848143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.848261 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:35.848363 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:35.848393 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 13:06:35.848452 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.224.65@KRBTEST.COM
I20260806 13:06:35.870785 15683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.864527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44138 (local address 127.13.224.65:46229)
0806 13:06:35.864579 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:35.864580 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:35.864661 (+ 81us) server_negotiation.cc:407] Connection header received
0806 13:06:35.864700 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:35.864701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:35.864709 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:35.864764 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:35.865305 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.865790 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.866516 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.866693 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.866745 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.868715 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:35.868716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:35.868717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:35.868724 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:35.869745 (+ 1021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.870066 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.870067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.870067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.870117 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.870288 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.870290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.870290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.870379 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:35.870462 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:35.870625 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 13:06:35.870696 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[14945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.65@KRBTEST.COM
W20260806 13:06:35.891187 15280 tablet.cc:2369] T 2e407e299d034a8983a0b922c7d4cf8c P 7b7f5e339d7a43588e7f2cd002b686f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:06:35.891760 15683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:35.885520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44150 (local address 127.13.224.65:46229)
0806 13:06:35.885585 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:35.885587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:35.885643 (+ 56us) server_negotiation.cc:407] Connection header received
0806 13:06:35.885709 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:35.885710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:35.885720 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:35.885766 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:35.886358 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.886814 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.887504 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:35.887683 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:35.887742 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:35.889678 (+ 1936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:35.889680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:35.889680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:35.889688 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:35.890766 (+ 1078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.891084 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.891085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.891085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.891133 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:35.891305 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:35.891306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:35.891306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:35.891394 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:35.891490 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:35.891611 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 13:06:35.891677 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 13:06:35.892397 15155 tablet_service.cc:2908] Rejecting scan request for tablet 2e407e299d034a8983a0b922c7d4cf8c: Uninitialized: clean time has not yet been initialized
I20260806 13:06:35.893249 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 15096
I20260806 13:06:35.906771 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 15285
I20260806 13:06:35.915413 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 15483
I20260806 13:06:35.924778 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 14966
2026-08-06T13:06:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2084 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[15714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[15714](info): set up 2 sockets
Aug 06 13:06:35 dist-test-slave-q9h9 krb5kdc[15714](info): commencing operation
krb5kdc: starting...
W20260806 13:06:37.037546 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:06:37 dist-test-slave-q9h9 krb5kdc[15714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:06:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:06:37.145015 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:45169
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:45169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:37.224813 15729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:37.224997 15729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:37.225028 15729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:37.226162 15729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:37.226199 15729 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 13:06:37.226218 15729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:37.226234 15729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:37.226250 15729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:37.226274 15729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:06:37.227716 15729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:45169
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:45169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:37.227942 15729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:37.228148 15729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:37.232580 15737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:37.232594 15739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:37.232596 15742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:37.232673 15729 server_base.cc:1034] running on GCE node
I20260806 13:06:37.233042 15729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:37.233317 15729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:37.234448 15729 hybrid_clock.cc:648] HybridClock initialized: now 1786021597234439 us; error 27 us; skew 500 ppm
Aug 06 13:06:37 dist-test-slave-q9h9 krb5kdc[15714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021597, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:37.239377 15729 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:06:37.239769 15729 webserver.cc:460] Webserver started at http://127.13.224.126:43267/ using document root <none> and password file <none>
I20260806 13:06:37.239950 15729 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:37.239987 15729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:37.240073 15729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:37.240981 15729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "c82cc65a483e408cbb38c307e139d558"
format_stamp: "Formatted at 2026-08-06 13:06:37 on dist-test-slave-q9h9"
server_key: "70f8c17d8780e6dc5ff7293c0215b859"
server_key_iv: "6293fbc5bbb8f9e1c5ecb3e4c71e47db"
server_key_version: "encryptionkey@0"
I20260806 13:06:37.241266 15729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "c82cc65a483e408cbb38c307e139d558"
format_stamp: "Formatted at 2026-08-06 13:06:37 on dist-test-slave-q9h9"
server_key: "70f8c17d8780e6dc5ff7293c0215b859"
server_key_iv: "6293fbc5bbb8f9e1c5ecb3e4c71e47db"
server_key_version: "encryptionkey@0"
I20260806 13:06:37.242496 15729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:06:37.243129 15753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:37.243290 15729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:37.243362 15729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "c82cc65a483e408cbb38c307e139d558"
format_stamp: "Formatted at 2026-08-06 13:06:37 on dist-test-slave-q9h9"
server_key: "70f8c17d8780e6dc5ff7293c0215b859"
server_key_iv: "6293fbc5bbb8f9e1c5ecb3e4c71e47db"
server_key_version: "encryptionkey@0"
I20260806 13:06:37.243412 15729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:37.255133 15729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:37.256924 15729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:37.257038 15729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:37.261502 15729 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:45169
I20260806 13:06:37.261561 15843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:45169 every 8 connection(s)
I20260806 13:06:37.261837 15729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:37.262486 15845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:37.264569 15845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558: Bootstrap starting.
I20260806 13:06:37.265127 15845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:37.265379 15845 log.cc:824] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:37.266036 15845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558: No bootstrap required, opened a new log
I20260806 13:06:37.268107 15845 raft_consensus.cc:348] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c82cc65a483e408cbb38c307e139d558" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45169 } }
I20260806 13:06:37.268244 15845 raft_consensus.cc:374] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:37.268275 15845 raft_consensus.cc:729] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c82cc65a483e408cbb38c307e139d558, State: Initialized, Role: FOLLOWER
I20260806 13:06:37.268391 15845 consensus_queue.cc:260] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c82cc65a483e408cbb38c307e139d558" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45169 } }
I20260806 13:06:37.268451 15845 raft_consensus.cc:388] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:37.268481 15845 raft_consensus.cc:482] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:37.268522 15845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:37.269002 15845 raft_consensus.cc:504] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c82cc65a483e408cbb38c307e139d558" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45169 } }
I20260806 13:06:37.269094 15845 leader_election.cc:302] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c82cc65a483e408cbb38c307e139d558; no voters:
I20260806 13:06:37.269138 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 15729
I20260806 13:06:37.269225 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:37.269263 15845 leader_election.cc:288] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:37.269315 15848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:37.269385 14209 external_mini_cluster.cc:1428] Setting key 5ad2eb57adaaccf675dd0316283f9273
I20260806 13:06:37.269459 15848 raft_consensus.cc:686] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [term 1 LEADER]: Becoming Leader. State: Replica: c82cc65a483e408cbb38c307e139d558, State: Running, Role: LEADER
I20260806 13:06:37.269541 15848 consensus_queue.cc:237] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c82cc65a483e408cbb38c307e139d558" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45169 } }
I20260806 13:06:37.269567 15845 sys_catalog.cc:564] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:37.270073 15851 sys_catalog.cc:455] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c82cc65a483e408cbb38c307e139d558. Latest consensus state: current_term: 1 leader_uuid: "c82cc65a483e408cbb38c307e139d558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c82cc65a483e408cbb38c307e139d558" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45169 } } }
I20260806 13:06:37.270077 15850 sys_catalog.cc:455] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c82cc65a483e408cbb38c307e139d558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c82cc65a483e408cbb38c307e139d558" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45169 } } }
I20260806 13:06:37.270176 15851 sys_catalog.cc:458] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:37.270198 15850 sys_catalog.cc:458] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:37.270645 15868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:37.271152 15868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:37.272768 15868 catalog_manager.cc:1347] Generated new cluster ID: ada11d7dd18d4309bda84325a1a3d217
I20260806 13:06:37.272820 15868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:06:37 dist-test-slave-q9h9 krb5kdc[15714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:37.279609 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.269820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57904 (local address 127.13.224.126:45169)
0806 13:06:37.270457 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.270467 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.270499 (+ 32us) server_negotiation.cc:407] Connection header received
0806 13:06:37.270618 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.270625 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.270699 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.270884 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:37.271613 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.272828 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.273554 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.273947 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.274057 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.276178 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:37.276181 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:37.276182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:37.276193 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:37.278155 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.278472 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.278475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.278475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.278571 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.278762 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.278763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.278763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.278981 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:37.279123 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.279241 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.279455 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":62,"threads_started":1}
I20260806 13:06:37.289700 15868 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:37.290267 15868 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:37.296921 15868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558: Generated new TSK 0
I20260806 13:06:37.297058 15868 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:06:37.324175 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:45169
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:37.403873 15881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:37.404059 15881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:37.404083 15881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:37.405216 15881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:37.405256 15881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:37.405275 15881 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 13:06:37.405292 15881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:06:37.406704 15881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.224.126:45169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:37.406906 15881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:37.407112 15881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:37.411895 15886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:37.412145 15890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:37.411888 15887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:37.412233 15881 server_base.cc:1034] running on GCE node
I20260806 13:06:37.412451 15881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:37.412678 15881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:37.413820 15881 hybrid_clock.cc:648] HybridClock initialized: now 1786021597413808 us; error 31 us; skew 500 ppm
Aug 06 13:06:37 dist-test-slave-q9h9 krb5kdc[15714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021597, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:37.416200 15881 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:06:37.416672 15881 webserver.cc:460] Webserver started at http://127.13.224.65:38281/ using document root <none> and password file <none>
I20260806 13:06:37.416867 15881 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:37.416914 15881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:37.417038 15881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:37.417977 15881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2"
format_stamp: "Formatted at 2026-08-06 13:06:37 on dist-test-slave-q9h9"
server_key: "1314d61b1de1d4b7b8c498d5ea2dbcc7"
server_key_iv: "b3463004659cd9956f6ff6364f3186d1"
server_key_version: "encryptionkey@0"
I20260806 13:06:37.418363 15881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2"
format_stamp: "Formatted at 2026-08-06 13:06:37 on dist-test-slave-q9h9"
server_key: "1314d61b1de1d4b7b8c498d5ea2dbcc7"
server_key_iv: "b3463004659cd9956f6ff6364f3186d1"
server_key_version: "encryptionkey@0"
I20260806 13:06:37.419605 15881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:06:37.420315 15899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:37.420498 15881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:37.420580 15881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2"
format_stamp: "Formatted at 2026-08-06 13:06:37 on dist-test-slave-q9h9"
server_key: "1314d61b1de1d4b7b8c498d5ea2dbcc7"
server_key_iv: "b3463004659cd9956f6ff6364f3186d1"
server_key_version: "encryptionkey@0"
I20260806 13:06:37.420656 15881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:37.435304 15881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:37.437693 15881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:37.437819 15881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:37.438437 15881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:37.438494 15881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:37.438522 15881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:37.438534 15881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:37.445326 15881 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:43927
I20260806 13:06:37.445406 16065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:43927 every 8 connection(s)
I20260806 13:06:37.445770 15881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
Aug 06 13:06:37 dist-test-slave-q9h9 krb5kdc[15714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021597, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:37.448724 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 15881
I20260806 13:06:37.448827 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:06:37.448985 14209 external_mini_cluster.cc:1428] Setting key 393efc3137cbfe9d92eeb2ffc00796ed
I20260806 13:06:37.449867 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.438882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:44999 (local address 127.13.224.126:45169)
0806 13:06:37.438958 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.438961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.439165 (+ 204us) server_negotiation.cc:407] Connection header received
0806 13:06:37.439783 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.439785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.439810 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.439897 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:37.440806 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.441682 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.443269 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.443640 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.443726 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.446519 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:37.446521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:37.446522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:37.446532 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:37.448378 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.448846 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.448848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.448848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.448918 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.449198 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.449199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.449199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.449337 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:37.449397 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.449683 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.449776 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 13:06:37.450073 15911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.438843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:45169 (local address 127.13.224.65:44999)
0806 13:06:37.439009 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:37.439052 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:37.439574 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:37.439999 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:37.440004 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:37.440385 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:37.440672 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.440680 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.441815 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.441816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.443161 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.443166 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.443765 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.443766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.443883 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.444506 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:37.444517 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:37.446393 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:37.448468 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:37.448476 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:37.448478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:37.448749 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:37.448988 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:37.448989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:37.448990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:37.449132 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:37.449403 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:37.449410 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:37.449552 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 13:06:37.449645 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 13:06:37.449823 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260806 13:06:37.451771 16066 heartbeater.cc:347] Connected to a master server at 127.13.224.126:45169
I20260806 13:06:37.451862 16066 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:37.452062 16066 heartbeater.cc:511] Master 127.13.224.126:45169 requested a full tablet report, sending...
I20260806 13:06:37.452632 15773 ts_manager.cc:194] Registered new tserver with Master: 7bd6a65095bc467ebe16c0cc8d6dc7d2 (127.13.224.65:43927)
I20260806 13:06:37.453526 15773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:44999
I20260806 13:06:37.458406 16080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.452919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:45169 (local address 127.13.224.65:52967)
0806 13:06:37.453062 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:37.453072 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:37.453127 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:37.453237 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:37.453238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:37.453443 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:37.453508 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.453510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.454297 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.454298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.454833 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.454835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.455171 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.455172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.455250 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.455665 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:37.455670 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:37.455954 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:37.457440 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:37.457441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:37.457442 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:37.457603 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:37.457794 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:37.457795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:37.457795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:37.457853 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:37.458092 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:37.458093 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:37.458141 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 13:06:37.458202 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 13:06:37.458309 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260806 13:06:37.458401 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.452997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:52967 (local address 127.13.224.126:45169)
0806 13:06:37.453053 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.453054 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.453113 (+ 59us) server_negotiation.cc:407] Connection header received
0806 13:06:37.453175 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.453176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.453184 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.453233 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:37.453573 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.454193 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.454912 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.455105 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.455159 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.456040 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:37.456042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:37.456042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:37.456049 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:37.457363 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.457669 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.457670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.457671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.457734 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.457908 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.457909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.457909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.458039 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:37.458101 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.458237 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.458308 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:06:37.477629 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:45169
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:37.559832 16082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:37.560029 16082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:37.560050 16082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:37.561182 16082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:37.561221 16082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:37.561249 16082 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 13:06:37.561264 16082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:06:37.562801 16082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.224.126:45169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:37.563045 16082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:37.563324 16082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:37.568316 16094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:37.568374 16090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:37.568565 16097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:37.568578 16082 server_base.cc:1034] running on GCE node
I20260806 13:06:37.568795 16082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:37.569033 16082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:37.570171 16082 hybrid_clock.cc:648] HybridClock initialized: now 1786021597570158 us; error 36 us; skew 500 ppm
Aug 06 13:06:37 dist-test-slave-q9h9 krb5kdc[15714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021597, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:37.572750 16082 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:06:37.573233 16082 webserver.cc:460] Webserver started at http://127.13.224.66:33351/ using document root <none> and password file <none>
I20260806 13:06:37.573439 16082 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:37.573488 16082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:37.573618 16082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:37.574740 16082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "23c96c2328394a0986b8654b9b458371"
format_stamp: "Formatted at 2026-08-06 13:06:37 on dist-test-slave-q9h9"
server_key: "55bce9c68d95636f328bd07f996063be"
server_key_iv: "fb8ba063c95f0809fd3993952f99aa1a"
server_key_version: "encryptionkey@0"
I20260806 13:06:37.575240 16082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "23c96c2328394a0986b8654b9b458371"
format_stamp: "Formatted at 2026-08-06 13:06:37 on dist-test-slave-q9h9"
server_key: "55bce9c68d95636f328bd07f996063be"
server_key_iv: "fb8ba063c95f0809fd3993952f99aa1a"
server_key_version: "encryptionkey@0"
I20260806 13:06:37.576582 16082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:06:37.577410 16106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:37.577612 16082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:06:37.577708 16082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "23c96c2328394a0986b8654b9b458371"
format_stamp: "Formatted at 2026-08-06 13:06:37 on dist-test-slave-q9h9"
server_key: "55bce9c68d95636f328bd07f996063be"
server_key_iv: "fb8ba063c95f0809fd3993952f99aa1a"
server_key_version: "encryptionkey@0"
I20260806 13:06:37.577775 16082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:37.588689 16082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:37.590930 16082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:37.591079 16082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:37.591629 16082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:37.591670 16082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:37.591701 16082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:37.591719 16082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 13:06:37 dist-test-slave-q9h9 krb5kdc[15714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021597, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:37.598974 16082 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:46103
I20260806 13:06:37.599197 16265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:46103 every 8 connection(s)
I20260806 13:06:37.599469 16082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:06:37.601994 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.592032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:36491 (local address 127.13.224.126:45169)
0806 13:06:37.592085 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.592087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.592325 (+ 238us) server_negotiation.cc:407] Connection header received
0806 13:06:37.592841 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.592843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.592858 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.592911 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:37.593719 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.594489 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.595611 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.595815 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.595873 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.599039 (+ 3166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:37.599041 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:37.599042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:37.599224 (+ 182us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:37.600660 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.601042 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.601043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.601044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.601107 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.601343 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.601344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.601345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.601460 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:37.601518 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.601820 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.601904 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 13:06:37.602172 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.592033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:45169 (local address 127.13.224.66:36491)
0806 13:06:37.592188 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:37.592231 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:37.592733 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:37.592999 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:37.593002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:37.593327 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:37.593597 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.593603 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.594606 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.594607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.595519 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.595522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.595892 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.595893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.596028 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.596517 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:37.596528 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:37.598943 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:37.600748 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:37.600755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:37.600757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:37.600967 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:37.601169 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:37.601170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:37.601171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:37.601274 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:37.601524 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:37.601531 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:37.601667 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 13:06:37.601770 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 13:06:37.601937 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260806 13:06:37.602638 16266 heartbeater.cc:347] Connected to a master server at 127.13.224.126:45169
I20260806 13:06:37.602717 16266 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:37.602890 16266 heartbeater.cc:511] Master 127.13.224.126:45169 requested a full tablet report, sending...
I20260806 13:06:37.603319 15773 ts_manager.cc:194] Registered new tserver with Master: 23c96c2328394a0986b8654b9b458371 (127.13.224.66:46103)
I20260806 13:06:37.603688 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 16082
I20260806 13:06:37.603787 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:06:37.603811 15773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:36491
I20260806 13:06:37.604041 14209 external_mini_cluster.cc:1428] Setting key 7f96c3eca7bf494518a1fa55b34a4994
I20260806 13:06:37.609354 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.603677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:44743 (local address 127.13.224.126:45169)
0806 13:06:37.603715 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.603716 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.603849 (+ 133us) server_negotiation.cc:407] Connection header received
0806 13:06:37.603936 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.603937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.603947 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.603974 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:37.604396 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.604956 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.605699 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.605951 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.606040 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.606905 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:37.606906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:37.606906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:37.606912 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:37.608119 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.608615 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.608615 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.608616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.608663 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.608865 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.608865 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.608865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.608987 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:37.609035 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.609201 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.609288 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:06:37.609390 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.603625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:45169 (local address 127.13.224.66:44743)
0806 13:06:37.603791 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:37.603801 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:37.603873 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:37.604030 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:37.604031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:37.604247 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:37.604315 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.604317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.605059 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.605060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.605604 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.605605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.606049 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.606049 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.606143 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.606496 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:37.606501 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:37.606819 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:37.608194 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:37.608195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:37.608195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:37.608386 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:37.608733 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:37.608734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:37.608734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:37.608800 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:37.609052 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:37.609053 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:37.609100 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 13:06:37.609177 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 13:06:37.609278 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:06:37.630785 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:45169
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:37.713347 16279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:37.713552 16279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:37.713575 16279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:37.714740 16279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:37.714781 16279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:37.714833 16279 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 13:06:37.714864 16279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:06:37.716301 16279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.224.126:45169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:37.716512 16279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:37.716737 16279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:37.721719 16288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:37.721810 16287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:37.721901 16290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:37.721856 16279 server_base.cc:1034] running on GCE node
I20260806 13:06:37.722126 16279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:37.722355 16279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:37.723498 16279 hybrid_clock.cc:648] HybridClock initialized: now 1786021597723489 us; error 29 us; skew 500 ppm
Aug 06 13:06:37 dist-test-slave-q9h9 krb5kdc[15714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021597, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:37.726042 16279 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:06:37.726488 16279 webserver.cc:460] Webserver started at http://127.13.224.67:40327/ using document root <none> and password file <none>
I20260806 13:06:37.726713 16279 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:37.726760 16279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:37.726877 16279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:37.727900 16279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "b9064d4591d7499bb26d9941e28a7ab0"
format_stamp: "Formatted at 2026-08-06 13:06:37 on dist-test-slave-q9h9"
server_key: "018b6109406ae63e298aadc2b9fca070"
server_key_iv: "413ba3cda088d97f601455a702ee4ec2"
server_key_version: "encryptionkey@0"
I20260806 13:06:37.728232 16279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "b9064d4591d7499bb26d9941e28a7ab0"
format_stamp: "Formatted at 2026-08-06 13:06:37 on dist-test-slave-q9h9"
server_key: "018b6109406ae63e298aadc2b9fca070"
server_key_iv: "413ba3cda088d97f601455a702ee4ec2"
server_key_version: "encryptionkey@0"
I20260806 13:06:37.729470 16279 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 13:06:37.730213 16301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:37.730386 16279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 13:06:37.730461 16279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "b9064d4591d7499bb26d9941e28a7ab0"
format_stamp: "Formatted at 2026-08-06 13:06:37 on dist-test-slave-q9h9"
server_key: "018b6109406ae63e298aadc2b9fca070"
server_key_iv: "413ba3cda088d97f601455a702ee4ec2"
server_key_version: "encryptionkey@0"
I20260806 13:06:37.730523 16279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:37.759665 16279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:37.761505 16279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:37.761654 16279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:37.762300 16279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:37.762352 16279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:37.762382 16279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:37.762410 16279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 13:06:37 dist-test-slave-q9h9 krb5kdc[15714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021597, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:37.769410 16279 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:34307
I20260806 13:06:37.769502 16470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:34307 every 8 connection(s)
I20260806 13:06:37.769830 16279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:06:37.772256 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.762845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:41553 (local address 127.13.224.126:45169)
0806 13:06:37.762919 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.762921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.763133 (+ 212us) server_negotiation.cc:407] Connection header received
0806 13:06:37.763606 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.763607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.763621 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.763692 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:37.764368 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.765120 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.766265 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.766494 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.766551 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.769557 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:37.769559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:37.769559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:37.769567 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:37.771173 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.771528 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.771529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.771529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.771585 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.771781 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.771781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.771782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.771880 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:37.771926 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.772133 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.772191 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 13:06:37.772465 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.762804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:45169 (local address 127.13.224.67:41553)
0806 13:06:37.762992 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:37.763033 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:37.763517 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:37.763759 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:37.763764 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:37.764061 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:37.764285 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.764290 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.765209 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.765210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.766169 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.766172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.766574 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.766576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.766704 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.767174 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:37.767185 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:37.769467 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:37.771243 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:37.771248 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:37.771250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:37.771477 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:37.771640 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:37.771640 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:37.771640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:37.771729 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:37.771927 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:37.771930 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:37.772043 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 13:06:37.772120 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 13:06:37.772236 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
I20260806 13:06:37.772948 16473 heartbeater.cc:347] Connected to a master server at 127.13.224.126:45169
I20260806 13:06:37.773025 16473 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:37.773196 16473 heartbeater.cc:511] Master 127.13.224.126:45169 requested a full tablet report, sending...
I20260806 13:06:37.773557 15774 ts_manager.cc:194] Registered new tserver with Master: b9064d4591d7499bb26d9941e28a7ab0 (127.13.224.67:34307)
I20260806 13:06:37.774000 15774 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:41553
I20260806 13:06:37.776573 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 16279
I20260806 13:06:37.776664 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:06:37.776832 14209 external_mini_cluster.cc:1428] Setting key 2ba14b236a40cc1403a087e893d68a5a
I20260806 13:06:37.777684 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:06:37.779578 16479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.774046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:45169 (local address 127.13.224.67:42583)
0806 13:06:37.774207 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:37.774217 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:37.774280 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:37.774390 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:37.774391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:37.774544 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:37.774601 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.774602 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.775277 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.775277 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.775785 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.775787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.776128 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.776129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.776179 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.776538 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:37.776542 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:37.776892 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:37.778550 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:37.778551 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:37.778551 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:37.778783 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:37.779009 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:37.779010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:37.779010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:37.779093 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:37.779325 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:37.779325 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:37.779360 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 13:06:37.779402 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 13:06:37.779490 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":106,"threads_started":1}
I20260806 13:06:37.779834 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.774104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:42583 (local address 127.13.224.126:45169)
0806 13:06:37.774145 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.774147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.774270 (+ 123us) server_negotiation.cc:407] Connection header received
0806 13:06:37.774331 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.774331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.774339 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.774386 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:37.774671 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.775198 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.775882 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.776063 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.776127 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.776968 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:37.776969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:37.776969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:37.776976 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:37.778465 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.778858 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.778859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.778860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.778921 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.779149 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.779150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.779150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.779267 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:37.779331 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.779452 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.779521 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 13:06:37 dist-test-slave-q9h9 krb5kdc[15714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021597, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 13:06:37 dist-test-slave-q9h9 krb5kdc[15714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021597, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:37.801193 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.794715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57910 (local address 127.13.224.126:45169)
0806 13:06:37.794769 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.794770 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.794779 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:06:37.794815 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.794816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.794824 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.794871 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:37.795392 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.795876 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.796587 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.796786 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.796857 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.798975 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:37.798976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:37.798976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:37.798984 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:37.800168 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.800484 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.800485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.800486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.800537 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:37.800713 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:37.800714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:37.800715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:37.800814 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:37.800899 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.801031 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.801102 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 13:06:37.802996 15774 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:06:37.804715 15774 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:06:37.811129 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.806985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:43927 (local address 127.0.0.1:40202)
0806 13:06:37.807439 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:37.807447 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:37.807493 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:37.807696 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:37.807697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:37.807699 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:37.807752 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.807753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.808742 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.808743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.809937 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.809940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.810847 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.810848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.810920 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.810936 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 13:06:37.811000 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 13:06:37.811051 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
I20260806 13:06:37.811275 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.807264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.13.224.66:46103)
0806 13:06:37.807585 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.807589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.807599 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:06:37.807634 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.807636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.807650 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.807881 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:37.807947 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.808990 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.810236 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.810943 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.811012 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.811070 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.811132 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.811166 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":93,"threads_started":1}
I20260806 13:06:37.811384 16504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.807367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40202 (local address 127.13.224.65:43927)
0806 13:06:37.807542 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.807546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.807555 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:06:37.807591 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.807593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.807608 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.807680 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:37.807824 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.808649 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.810043 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.810753 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.810827 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.810885 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.811243 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.811286 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
I20260806 13:06:37.811646 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.806985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:46103 (local address 127.0.0.1:41870)
0806 13:06:37.807354 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:37.807363 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:37.807404 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:37.807775 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:37.807775 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:37.807778 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:37.807812 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.807813 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.809098 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.809099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.810160 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.810162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.811014 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.811015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.811064 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.811067 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:06:37.811110 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 13:06:37.811148 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":85,"threads_started":1}
I20260806 13:06:37.812068 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.806971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:34307 (local address 127.0.0.1:48728)
0806 13:06:37.807140 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:37.807180 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:37.807253 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:37.807574 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:37.807578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:37.807580 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:37.807645 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.807647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.810223 (+ 2576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.810224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.810897 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.810898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.811820 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.811821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.811892 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.811896 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:06:37.811941 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 13:06:37.811978 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":59,"threads_started":1}
I20260806 13:06:37.812089 15972 tablet_service.cc:1467] Processing CreateTablet for tablet 54c6c7688869437399bacae6bb8b4d09 (DEFAULT_TABLE table=test-table [id=bf6fcac352844e13872b49c245a48840]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:37.812295 16174 tablet_service.cc:1467] Processing CreateTablet for tablet 54c6c7688869437399bacae6bb8b4d09 (DEFAULT_TABLE table=test-table [id=bf6fcac352844e13872b49c245a48840]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:37.812383 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.807135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48728 (local address 127.13.224.67:34307)
0806 13:06:37.807327 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.807331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.807342 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:06:37.807383 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.807385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.807407 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.807492 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:37.807730 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.809005 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.810967 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.811672 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.811736 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.811786 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.812257 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.812295 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":108,"threads_started":1}
I20260806 13:06:37.812477 15972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54c6c7688869437399bacae6bb8b4d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:37.812594 16174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54c6c7688869437399bacae6bb8b4d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:37.812986 16365 tablet_service.cc:1467] Processing CreateTablet for tablet 54c6c7688869437399bacae6bb8b4d09 (DEFAULT_TABLE table=test-table [id=bf6fcac352844e13872b49c245a48840]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:37.813318 16365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54c6c7688869437399bacae6bb8b4d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:37.815405 16505 tablet_bootstrap.cc:492] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Bootstrap starting.
I20260806 13:06:37.815969 16505 tablet_bootstrap.cc:654] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:37.816192 16506 tablet_bootstrap.cc:492] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371: Bootstrap starting.
I20260806 13:06:37.816290 16505 log.cc:824] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:37.816735 16506 tablet_bootstrap.cc:654] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:37.817060 16506 log.cc:824] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:37.817379 16507 tablet_bootstrap.cc:492] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0: Bootstrap starting.
I20260806 13:06:37.817745 16506 tablet_bootstrap.cc:492] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371: No bootstrap required, opened a new log
I20260806 13:06:37.817806 16506 ts_tablet_manager.cc:1397] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:06:37.818120 16507 tablet_bootstrap.cc:654] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:37.818395 16507 log.cc:824] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:37.819062 16505 tablet_bootstrap.cc:492] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2: No bootstrap required, opened a new log
I20260806 13:06:37.819118 16505 ts_tablet_manager.cc:1397] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 13:06:37.819183 16507 tablet_bootstrap.cc:492] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0: No bootstrap required, opened a new log
I20260806 13:06:37.819274 16507 ts_tablet_manager.cc:1397] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 13:06:37.819572 16506 raft_consensus.cc:348] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } }
I20260806 13:06:37.819742 16506 raft_consensus.cc:374] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:37.819777 16506 raft_consensus.cc:729] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c96c2328394a0986b8654b9b458371, State: Initialized, Role: FOLLOWER
I20260806 13:06:37.819908 16506 consensus_queue.cc:260] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } }
I20260806 13:06:37.820181 16506 ts_tablet_manager.cc:1428] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 13:06:37.820214 16266 heartbeater.cc:503] Master 127.13.224.126:45169 was elected leader, sending a full tablet report...
I20260806 13:06:37.820875 16505 raft_consensus.cc:348] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } }
I20260806 13:06:37.821022 16505 raft_consensus.cc:374] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:37.821067 16505 raft_consensus.cc:729] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bd6a65095bc467ebe16c0cc8d6dc7d2, State: Initialized, Role: FOLLOWER
I20260806 13:06:37.821177 16505 consensus_queue.cc:260] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } }
I20260806 13:06:37.821203 16507 raft_consensus.cc:348] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } }
I20260806 13:06:37.821349 16507 raft_consensus.cc:374] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:37.821385 16507 raft_consensus.cc:729] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9064d4591d7499bb26d9941e28a7ab0, State: Initialized, Role: FOLLOWER
I20260806 13:06:37.821455 16505 ts_tablet_manager.cc:1428] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 13:06:37.821506 16066 heartbeater.cc:503] Master 127.13.224.126:45169 was elected leader, sending a full tablet report...
I20260806 13:06:37.821501 16507 consensus_queue.cc:260] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } }
I20260806 13:06:37.821765 16507 ts_tablet_manager.cc:1428] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 13:06:37.821792 16473 heartbeater.cc:503] Master 127.13.224.126:45169 was elected leader, sending a full tablet report...
W20260806 13:06:37.850162 16267 tablet.cc:2369] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:06:37.904100 16513 raft_consensus.cc:482] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:06:37.904263 16513 raft_consensus.cc:504] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } }
I20260806 13:06:37.904646 16513 leader_election.cc:288] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9064d4591d7499bb26d9941e28a7ab0 (127.13.224.67:34307), 7bd6a65095bc467ebe16c0cc8d6dc7d2 (127.13.224.65:43927)
I20260806 13:06:37.907577 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.904986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:54699 (local address 127.13.224.67:34307)
0806 13:06:37.905043 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.905045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.905182 (+ 137us) server_negotiation.cc:407] Connection header received
0806 13:06:37.905213 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.905214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.905224 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.905278 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:37.905432 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.905927 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.906869 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.907250 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.907319 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.907328 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.907476 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.907515 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 13:06:37.907577 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.904859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:43927 (local address 127.13.224.66:49529)
0806 13:06:37.904921 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:37.904937 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:37.904989 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:37.905117 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:37.905118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:37.905120 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:37.905200 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.905202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.905918 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.905920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.906733 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.906735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.907292 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.907293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.907391 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.907395 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:06:37.907436 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 13:06:37.907489 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":15}
I20260806 13:06:37.907589 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.904888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:34307 (local address 127.13.224.66:54699)
0806 13:06:37.905101 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:37.905109 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:37.905161 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:37.905293 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:37.905294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:37.905295 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:37.905370 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.905372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.906010 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.906011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.906767 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:37.906769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.907343 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.907344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:37.907403 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.907406 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:06:37.907456 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 13:06:37.907494 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":124,"threads_started":1}
I20260806 13:06:37.907583 16504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.904952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:49529 (local address 127.13.224.65:43927)
0806 13:06:37.905002 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.905004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.905013 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:06:37.905043 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.905044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.905053 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.905107 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:37.905286 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.905833 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.906832 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.907201 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.907258 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.907266 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.907465 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.907508 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 13:06:37.907950 16394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54c6c7688869437399bacae6bb8b4d09" candidate_uuid: "23c96c2328394a0986b8654b9b458371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9064d4591d7499bb26d9941e28a7ab0" is_pre_election: true
I20260806 13:06:37.908017 16012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54c6c7688869437399bacae6bb8b4d09" candidate_uuid: "23c96c2328394a0986b8654b9b458371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" is_pre_election: true
I20260806 13:06:37.908099 16394 raft_consensus.cc:2454] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23c96c2328394a0986b8654b9b458371 in term 0.
I20260806 13:06:37.908177 16012 raft_consensus.cc:2454] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23c96c2328394a0986b8654b9b458371 in term 0.
I20260806 13:06:37.908360 16112 leader_election.cc:302] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23c96c2328394a0986b8654b9b458371, b9064d4591d7499bb26d9941e28a7ab0; no voters:
I20260806 13:06:37.908494 16513 raft_consensus.cc:2781] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:06:37.908561 16513 raft_consensus.cc:482] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:06:37.908579 16513 raft_consensus.cc:3037] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:37.909583 16513 raft_consensus.cc:504] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } }
I20260806 13:06:37.909704 16513 leader_election.cc:288] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [CANDIDATE]: Term 1 election: Requested vote from peers b9064d4591d7499bb26d9941e28a7ab0 (127.13.224.67:34307), 7bd6a65095bc467ebe16c0cc8d6dc7d2 (127.13.224.65:43927)
I20260806 13:06:37.909883 16394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54c6c7688869437399bacae6bb8b4d09" candidate_uuid: "23c96c2328394a0986b8654b9b458371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9064d4591d7499bb26d9941e28a7ab0"
I20260806 13:06:37.909896 16012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54c6c7688869437399bacae6bb8b4d09" candidate_uuid: "23c96c2328394a0986b8654b9b458371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2"
I20260806 13:06:37.909948 16394 raft_consensus.cc:3037] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:37.909972 16012 raft_consensus.cc:3037] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:37.910668 16394 raft_consensus.cc:2454] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23c96c2328394a0986b8654b9b458371 in term 1.
I20260806 13:06:37.910725 16012 raft_consensus.cc:2454] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23c96c2328394a0986b8654b9b458371 in term 1.
I20260806 13:06:37.910853 16112 leader_election.cc:302] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23c96c2328394a0986b8654b9b458371, b9064d4591d7499bb26d9941e28a7ab0; no voters:
I20260806 13:06:37.910972 16513 raft_consensus.cc:2781] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:37.911123 16513 raft_consensus.cc:686] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [term 1 LEADER]: Becoming Leader. State: Replica: 23c96c2328394a0986b8654b9b458371, State: Running, Role: LEADER
I20260806 13:06:37.911224 16513 consensus_queue.cc:237] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } }
I20260806 13:06:37.912060 15774 catalog_manager.cc:5495] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23c96c2328394a0986b8654b9b458371 (127.13.224.66). New cstate: current_term: 1 leader_uuid: "23c96c2328394a0986b8654b9b458371" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } health_report { overall_health: UNKNOWN } } }
I20260806 13:06:37.929591 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:37.927201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41534 (local address 127.13.224.66:46103)
0806 13:06:37.927273 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:37.927275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:37.927284 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:06:37.927369 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:37.927370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:37.927389 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:37.927442 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:06:37.927584 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.928054 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.928818 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:37.928999 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:37.929052 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:37.929264 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:06:37.929350 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:06:37.929415 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:37.929471 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 13:06:37.929492 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 13:06:37.931360 16012 raft_consensus.cc:1261] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 1 FOLLOWER]: Refusing update from remote peer 23c96c2328394a0986b8654b9b458371: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:37.931666 16518 consensus_queue.cc:1035] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:37.933276 16394 raft_consensus.cc:1261] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0 [term 1 FOLLOWER]: Refusing update from remote peer 23c96c2328394a0986b8654b9b458371: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:37.933548 16513 consensus_queue.cc:1035] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:37.934453 16526 mvcc.cc:205] Tried to move back new op lower bound from 7315544465124089856 to 7315544465044910080. Current Snapshot: MvccSnapshot[applied={T|T < 7315544465124089856}]
I20260806 13:06:37.938388 15774 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:37.938522 15774 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:37.939441 15774 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558: Sending DeleteTablet for 3 replicas of tablet 54c6c7688869437399bacae6bb8b4d09
I20260806 13:06:37.939896 16365 tablet_service.cc:1514] Processing DeleteTablet for tablet 54c6c7688869437399bacae6bb8b4d09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:37 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:48728
I20260806 13:06:37.940099 16174 tablet_service.cc:1514] Processing DeleteTablet for tablet 54c6c7688869437399bacae6bb8b4d09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:37 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:41870
I20260806 13:06:37.940105 15774 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:06:37.940246 15774 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:06:37.940425 16536 tablet_replica.cc:330] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0: stopping tablet replica
I20260806 13:06:37.940533 16537 tablet_replica.cc:330] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371: stopping tablet replica
I20260806 13:06:37.940636 16537 raft_consensus.cc:2229] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:06:37.940732 16536 raft_consensus.cc:2229] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:37.940865 16537 raft_consensus.cc:2258] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:37.940964 16536 raft_consensus.cc:2258] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:37.941478 16365 tablet_service.cc:1467] Processing CreateTablet for tablet 92621bd8608943d69f7b8a40d901c1fa (DEFAULT_TABLE table=test-table [id=557399158f0c44f989ffc6c81a8b0fe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:37.941574 16365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92621bd8608943d69f7b8a40d901c1fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:37.941597 15971 tablet_service.cc:1467] Processing CreateTablet for tablet 92621bd8608943d69f7b8a40d901c1fa (DEFAULT_TABLE table=test-table [id=557399158f0c44f989ffc6c81a8b0fe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:37.941740 15971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92621bd8608943d69f7b8a40d901c1fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:37.942534 16174 tablet_service.cc:1467] Processing CreateTablet for tablet 92621bd8608943d69f7b8a40d901c1fa (DEFAULT_TABLE table=test-table [id=557399158f0c44f989ffc6c81a8b0fe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:37.942620 16174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92621bd8608943d69f7b8a40d901c1fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:37.942767 15972 tablet_service.cc:1514] Processing DeleteTablet for tablet 54c6c7688869437399bacae6bb8b4d09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:37 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:40202
I20260806 13:06:37.942965 16539 tablet_replica.cc:330] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2: stopping tablet replica
I20260806 13:06:37.943041 16539 raft_consensus.cc:2229] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:37.943127 16539 raft_consensus.cc:2258] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:37.943559 16539 ts_tablet_manager.cc:1905] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:37.943766 16505 tablet_bootstrap.cc:492] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Bootstrap starting.
I20260806 13:06:37.944058 16507 tablet_bootstrap.cc:492] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0: Bootstrap starting.
I20260806 13:06:37.944164 16505 tablet_bootstrap.cc:654] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:37.944631 16507 tablet_bootstrap.cc:654] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:37.945001 16506 tablet_bootstrap.cc:492] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371: Bootstrap starting.
I20260806 13:06:37.945186 16539 ts_tablet_manager.cc:1918] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:06:37.945251 16539 log.cc:1196] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/wal/wals/54c6c7688869437399bacae6bb8b4d09
I20260806 13:06:37.945263 16507 tablet_bootstrap.cc:492] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0: No bootstrap required, opened a new log
I20260806 13:06:37.945310 16507 ts_tablet_manager.cc:1397] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:37.945416 16506 tablet_bootstrap.cc:654] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:37.945457 16507 raft_consensus.cc:348] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:37.945535 16507 raft_consensus.cc:374] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:37.945556 16507 raft_consensus.cc:729] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9064d4591d7499bb26d9941e28a7ab0, State: Initialized, Role: FOLLOWER
I20260806 13:06:37.945580 16539 ts_tablet_manager.cc:1939] T 54c6c7688869437399bacae6bb8b4d09 P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Deleting consensus metadata
I20260806 13:06:37.945600 16507 consensus_queue.cc:260] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:37.945688 16507 ts_tablet_manager.cc:1428] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:37.946061 15756 catalog_manager.cc:4841] TS 7bd6a65095bc467ebe16c0cc8d6dc7d2 (127.13.224.65:43927): tablet 54c6c7688869437399bacae6bb8b4d09 (table test-table [id=bf6fcac352844e13872b49c245a48840]) successfully deleted
I20260806 13:06:37.946529 16536 ts_tablet_manager.cc:1905] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:37.946724 16537 ts_tablet_manager.cc:1905] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:37.948148 16537 ts_tablet_manager.cc:1918] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:06:37.948217 16537 log.cc:1196] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/54c6c7688869437399bacae6bb8b4d09
I20260806 13:06:37.948287 16506 tablet_bootstrap.cc:492] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371: No bootstrap required, opened a new log
I20260806 13:06:37.948328 16506 ts_tablet_manager.cc:1397] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 13:06:37.948091 16536 ts_tablet_manager.cc:1918] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:06:37.948467 16506 raft_consensus.cc:348] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:37.948545 16506 raft_consensus.cc:374] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:37.948573 16506 raft_consensus.cc:729] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c96c2328394a0986b8654b9b458371, State: Initialized, Role: FOLLOWER
I20260806 13:06:37.948625 16536 log.cc:1196] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/54c6c7688869437399bacae6bb8b4d09
I20260806 13:06:37.948621 16506 consensus_queue.cc:260] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:37.948721 16506 ts_tablet_manager.cc:1428] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:37.948462 16537 ts_tablet_manager.cc:1939] T 54c6c7688869437399bacae6bb8b4d09 P 23c96c2328394a0986b8654b9b458371: Deleting consensus metadata
I20260806 13:06:37.949003 16536 ts_tablet_manager.cc:1939] T 54c6c7688869437399bacae6bb8b4d09 P b9064d4591d7499bb26d9941e28a7ab0: Deleting consensus metadata
I20260806 13:06:37.949138 15755 catalog_manager.cc:4841] TS 23c96c2328394a0986b8654b9b458371 (127.13.224.66:46103): tablet 54c6c7688869437399bacae6bb8b4d09 (table test-table [id=bf6fcac352844e13872b49c245a48840]) successfully deleted
I20260806 13:06:37.949440 15757 catalog_manager.cc:4841] TS b9064d4591d7499bb26d9941e28a7ab0 (127.13.224.67:34307): tablet 54c6c7688869437399bacae6bb8b4d09 (table test-table [id=bf6fcac352844e13872b49c245a48840]) successfully deleted
I20260806 13:06:37.950393 16505 tablet_bootstrap.cc:492] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2: No bootstrap required, opened a new log
I20260806 13:06:37.950452 16505 ts_tablet_manager.cc:1397] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260806 13:06:37.950610 16505 raft_consensus.cc:348] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:37.950690 16505 raft_consensus.cc:374] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:37.950709 16505 raft_consensus.cc:729] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bd6a65095bc467ebe16c0cc8d6dc7d2, State: Initialized, Role: FOLLOWER
I20260806 13:06:37.950754 16505 consensus_queue.cc:260] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:37.950837 16505 ts_tablet_manager.cc:1428] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260806 13:06:38.020898 16474 tablet.cc:2369] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:06:38.086321 16515 raft_consensus.cc:482] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:06:38.086427 16515 raft_consensus.cc:504] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:38.086794 16515 leader_election.cc:288] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bd6a65095bc467ebe16c0cc8d6dc7d2 (127.13.224.65:43927), 23c96c2328394a0986b8654b9b458371 (127.13.224.66:46103)
I20260806 13:06:38.090632 16504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:38.087162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:36035 (local address 127.13.224.65:43927)
0806 13:06:38.087203 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:38.087205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:38.087234 (+ 29us) server_negotiation.cc:407] Connection header received
0806 13:06:38.087284 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:38.087285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:38.087293 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:38.087345 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:38.087496 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.088224 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.089738 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.090249 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.090313 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:38.090322 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:38.090519 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 13:06:38.090559 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:06:38.090634 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:38.087078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:51367 (local address 127.13.224.66:46103)
0806 13:06:38.087130 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:38.087132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:38.087167 (+ 35us) server_negotiation.cc:407] Connection header received
0806 13:06:38.087200 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:38.087202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:38.087214 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:38.087271 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:38.087448 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.088209 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.089785 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.090279 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.090351 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:38.090361 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:38.090520 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 13:06:38.090560 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:06:38.090627 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:38.086996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:46103 (local address 127.13.224.67:51367)
0806 13:06:38.087059 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:38.087072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:38.087127 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:38.087293 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:38.087294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:38.087295 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:38.087376 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:38.087378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.088322 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.088323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:38.089653 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:38.089657 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.090373 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.090374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:38.090447 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:38.090450 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:06:38.090500 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 13:06:38.090546 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":33}
I20260806 13:06:38.090647 16545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:38.087043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:43927 (local address 127.13.224.67:36035)
0806 13:06:38.087179 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:38.087187 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:38.087235 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:38.087358 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:38.087358 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:38.087360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:38.087424 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:38.087425 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.088322 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.088323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:38.089654 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:38.089657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.090369 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.090371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:38.090440 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:38.090444 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:06:38.090489 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 13:06:38.090533 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":81,"thread_start_us":78,"threads_started":1}
I20260806 13:06:38.090880 16012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92621bd8608943d69f7b8a40d901c1fa" candidate_uuid: "b9064d4591d7499bb26d9941e28a7ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" is_pre_election: true
I20260806 13:06:38.090971 16012 raft_consensus.cc:2454] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9064d4591d7499bb26d9941e28a7ab0 in term 0.
I20260806 13:06:38.091193 16304 leader_election.cc:302] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bd6a65095bc467ebe16c0cc8d6dc7d2, b9064d4591d7499bb26d9941e28a7ab0; no voters:
I20260806 13:06:38.091172 16207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92621bd8608943d69f7b8a40d901c1fa" candidate_uuid: "b9064d4591d7499bb26d9941e28a7ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c96c2328394a0986b8654b9b458371" is_pre_election: true
I20260806 13:06:38.091317 16207 raft_consensus.cc:2454] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9064d4591d7499bb26d9941e28a7ab0 in term 0.
I20260806 13:06:38.091357 16515 raft_consensus.cc:2781] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:06:38.091410 16515 raft_consensus.cc:482] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:06:38.091430 16515 raft_consensus.cc:3037] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:38.092132 16515 raft_consensus.cc:504] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:38.092257 16515 leader_election.cc:288] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [CANDIDATE]: Term 1 election: Requested vote from peers 7bd6a65095bc467ebe16c0cc8d6dc7d2 (127.13.224.65:43927), 23c96c2328394a0986b8654b9b458371 (127.13.224.66:46103)
I20260806 13:06:38.092423 16012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92621bd8608943d69f7b8a40d901c1fa" candidate_uuid: "b9064d4591d7499bb26d9941e28a7ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2"
I20260806 13:06:38.092489 16012 raft_consensus.cc:3037] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:38.092469 16207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92621bd8608943d69f7b8a40d901c1fa" candidate_uuid: "b9064d4591d7499bb26d9941e28a7ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c96c2328394a0986b8654b9b458371"
I20260806 13:06:38.092554 16207 raft_consensus.cc:3037] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:38.093092 16012 raft_consensus.cc:2454] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9064d4591d7499bb26d9941e28a7ab0 in term 1.
I20260806 13:06:38.093240 16304 leader_election.cc:302] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bd6a65095bc467ebe16c0cc8d6dc7d2, b9064d4591d7499bb26d9941e28a7ab0; no voters:
I20260806 13:06:38.093243 16207 raft_consensus.cc:2454] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9064d4591d7499bb26d9941e28a7ab0 in term 1.
I20260806 13:06:38.093341 16515 raft_consensus.cc:2781] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:38.093488 16515 raft_consensus.cc:686] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [term 1 LEADER]: Becoming Leader. State: Replica: b9064d4591d7499bb26d9941e28a7ab0, State: Running, Role: LEADER
I20260806 13:06:38.093557 16515 consensus_queue.cc:237] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:38.094321 15773 catalog_manager.cc:5495] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9064d4591d7499bb26d9941e28a7ab0 (127.13.224.67). New cstate: current_term: 1 leader_uuid: "b9064d4591d7499bb26d9941e28a7ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } health_report { overall_health: HEALTHY } } }
Aug 06 13:06:38 dist-test-slave-q9h9 krb5kdc[15714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021597, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:38.149290 16554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:38.142745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:45169 (local address 127.0.0.1:34858)
0806 13:06:38.142911 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:38.142922 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:38.142993 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:38.143123 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:38.143124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:38.143447 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:38.143496 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:38.143498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.144171 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.144171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:38.144697 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:38.144698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.145030 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.145031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:38.145078 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:38.145587 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:38.145592 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:38.146967 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:38.148266 (+ 1299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:38.148274 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:38.148277 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:38.148460 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:38.148639 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:38.148639 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:38.148640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:38.148695 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:38.148928 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:38.148936 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:38.148987 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 13:06:38.149046 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 13:06:38.149165 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260806 13:06:38.149341 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:38.142810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34858 (local address 127.13.224.126:45169)
0806 13:06:38.142879 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:38.142881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:38.142991 (+ 110us) server_negotiation.cc:407] Connection header received
0806 13:06:38.143048 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:38.143049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:38.143061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:38.143108 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:38.143586 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.144088 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.144788 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.144962 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.145017 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:38.147061 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:38.147062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:38.147062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:38.147069 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:38.148180 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:38.148531 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:38.148532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:38.148533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:38.148580 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:38.148759 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:38.148760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:38.148760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:38.148865 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:38.148922 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:38.149080 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 13:06:38.149237 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":41}
I20260806 13:06:38.151293 15773 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:34858:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 13:06:38.151567 15773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:06:38.153132 15971 tablet_service.cc:1467] Processing CreateTablet for tablet 0ffcfa711f324e998cabc6986adbc40f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f41192565fa045d09382346be46b693c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 13:06:38.153231 16365 tablet_service.cc:1467] Processing CreateTablet for tablet 0ffcfa711f324e998cabc6986adbc40f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f41192565fa045d09382346be46b693c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 13:06:38.153355 15971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ffcfa711f324e998cabc6986adbc40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:38.153486 16365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ffcfa711f324e998cabc6986adbc40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:38.154593 16505 tablet_bootstrap.cc:492] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Bootstrap starting.
I20260806 13:06:38.155110 16505 tablet_bootstrap.cc:654] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:38.155426 16507 tablet_bootstrap.cc:492] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0: Bootstrap starting.
I20260806 13:06:38.155707 16505 tablet_bootstrap.cc:492] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2: No bootstrap required, opened a new log
I20260806 13:06:38.155748 16505 ts_tablet_manager.cc:1397] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 13:06:38.155797 16507 tablet_bootstrap.cc:654] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:38.155892 16505 raft_consensus.cc:348] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:38.155964 16505 raft_consensus.cc:374] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:38.155985 16505 raft_consensus.cc:729] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bd6a65095bc467ebe16c0cc8d6dc7d2, State: Initialized, Role: FOLLOWER
I20260806 13:06:38.153153 16174 tablet_service.cc:1467] Processing CreateTablet for tablet 0ffcfa711f324e998cabc6986adbc40f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f41192565fa045d09382346be46b693c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 13:06:38.156031 16505 consensus_queue.cc:260] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:38.156091 16174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ffcfa711f324e998cabc6986adbc40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:38.156128 16505 ts_tablet_manager.cc:1428] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:38.156236 16514 tablet_replica.cc:439] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } }
I20260806 13:06:38.156322 16514 tablet_replica.cc:442] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:06:38.156453 16507 tablet_bootstrap.cc:492] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0: No bootstrap required, opened a new log
I20260806 13:06:38.156490 16507 ts_tablet_manager.cc:1397] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:38.156627 16507 raft_consensus.cc:348] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:38.156697 16507 raft_consensus.cc:374] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:38.156724 16507 raft_consensus.cc:729] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9064d4591d7499bb26d9941e28a7ab0, State: Initialized, Role: FOLLOWER
I20260806 13:06:38.156785 16507 consensus_queue.cc:260] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:38.156878 16507 ts_tablet_manager.cc:1428] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:38.157004 16515 tablet_replica.cc:439] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } }
I20260806 13:06:38.157079 16515 tablet_replica.cc:442] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:06:38.157557 16506 tablet_bootstrap.cc:492] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371: Bootstrap starting.
I20260806 13:06:38.157902 16506 tablet_bootstrap.cc:654] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:38.158370 16506 tablet_bootstrap.cc:492] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371: No bootstrap required, opened a new log
I20260806 13:06:38.158411 16506 ts_tablet_manager.cc:1397] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 13:06:38.158488 16506 raft_consensus.cc:348] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:38.158535 16506 raft_consensus.cc:374] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:38.158546 16506 raft_consensus.cc:729] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c96c2328394a0986b8654b9b458371, State: Initialized, Role: FOLLOWER
I20260806 13:06:38.158577 16506 consensus_queue.cc:260] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:38.158643 16506 ts_tablet_manager.cc:1428] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:38.158710 16518 tablet_replica.cc:439] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } }
I20260806 13:06:38.158785 16518 tablet_replica.cc:442] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 13:06:38.196722 16067 tablet.cc:2369] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:06:38.204224 16515 raft_consensus.cc:482] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:06:38.204309 16515 raft_consensus.cc:504] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:38.204464 16515 leader_election.cc:288] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bd6a65095bc467ebe16c0cc8d6dc7d2 (127.13.224.65:43927), 23c96c2328394a0986b8654b9b458371 (127.13.224.66:46103)
I20260806 13:06:38.204674 16012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ffcfa711f324e998cabc6986adbc40f" candidate_uuid: "b9064d4591d7499bb26d9941e28a7ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" is_pre_election: true
I20260806 13:06:38.204681 16207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ffcfa711f324e998cabc6986adbc40f" candidate_uuid: "b9064d4591d7499bb26d9941e28a7ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c96c2328394a0986b8654b9b458371" is_pre_election: true
I20260806 13:06:38.204762 16012 raft_consensus.cc:2454] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9064d4591d7499bb26d9941e28a7ab0 in term 0.
I20260806 13:06:38.204773 16207 raft_consensus.cc:2454] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9064d4591d7499bb26d9941e28a7ab0 in term 0.
I20260806 13:06:38.204898 16304 leader_election.cc:302] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bd6a65095bc467ebe16c0cc8d6dc7d2, b9064d4591d7499bb26d9941e28a7ab0; no voters:
I20260806 13:06:38.205018 16515 raft_consensus.cc:2781] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:06:38.205057 16515 raft_consensus.cc:482] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:06:38.205070 16515 raft_consensus.cc:3037] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:38.205752 16515 raft_consensus.cc:504] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:38.205866 16515 leader_election.cc:288] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [CANDIDATE]: Term 1 election: Requested vote from peers 7bd6a65095bc467ebe16c0cc8d6dc7d2 (127.13.224.65:43927), 23c96c2328394a0986b8654b9b458371 (127.13.224.66:46103)
I20260806 13:06:38.206053 16012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ffcfa711f324e998cabc6986adbc40f" candidate_uuid: "b9064d4591d7499bb26d9941e28a7ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2"
I20260806 13:06:38.206111 16207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ffcfa711f324e998cabc6986adbc40f" candidate_uuid: "b9064d4591d7499bb26d9941e28a7ab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c96c2328394a0986b8654b9b458371"
I20260806 13:06:38.206172 16207 raft_consensus.cc:3037] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:38.206169 16012 raft_consensus.cc:3037] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:38.206928 16207 raft_consensus.cc:2454] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9064d4591d7499bb26d9941e28a7ab0 in term 1.
I20260806 13:06:38.206928 16012 raft_consensus.cc:2454] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9064d4591d7499bb26d9941e28a7ab0 in term 1.
I20260806 13:06:38.207103 16304 leader_election.cc:302] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bd6a65095bc467ebe16c0cc8d6dc7d2, b9064d4591d7499bb26d9941e28a7ab0; no voters:
I20260806 13:06:38.207186 16515 raft_consensus.cc:2781] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:38.207257 16515 raft_consensus.cc:686] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [term 1 LEADER]: Becoming Leader. State: Replica: b9064d4591d7499bb26d9941e28a7ab0, State: Running, Role: LEADER
I20260806 13:06:38.207304 16515 consensus_queue.cc:237] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } }
I20260806 13:06:38.207931 16546 tablet_replica.cc:439] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0: TxnStatusTablet state changed. Reason: New leader b9064d4591d7499bb26d9941e28a7ab0. Latest consensus state: current_term: 1 leader_uuid: "b9064d4591d7499bb26d9941e28a7ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } }
I20260806 13:06:38.208020 16546 tablet_replica.cc:442] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0: This TxnStatusTablet replica's current role is: LEADER
I20260806 13:06:38.208123 15774 catalog_manager.cc:5495] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9064d4591d7499bb26d9941e28a7ab0 (127.13.224.67). New cstate: current_term: 1 leader_uuid: "b9064d4591d7499bb26d9941e28a7ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } health_report { overall_health: HEALTHY } } }
I20260806 13:06:38.208261 16560 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 13:06:38.208384 16560 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 13:06:38.218750 16554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:38.216265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:34307 (local address 127.0.0.1:45540)
0806 13:06:38.216341 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:38.216350 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:38.216394 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:38.216553 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:38.216554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:38.216555 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 13:06:38.216609 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:38.216612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.217250 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.217251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:38.217815 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:38.217817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.218222 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.218223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:38.218297 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:38.218313 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 13:06:38.218578 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 13:06:38.218587 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 13:06:38.218631 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 13:06:38.218683 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 13:06:38.218729 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:38.216339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45540 (local address 127.13.224.67:34307)
0806 13:06:38.216401 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:38.216403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:38.216411 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:06:38.216474 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:38.216475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:38.216493 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:38.216549 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:06:38.216673 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.217153 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.217900 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.218143 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.218199 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:38.218413 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:06:38.218508 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:06:38.218574 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:38.218633 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 13:06:38.218658 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 13:06:38.220041 16207 raft_consensus.cc:1261] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371 [term 1 FOLLOWER]: Refusing update from remote peer b9064d4591d7499bb26d9941e28a7ab0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:38.220266 16012 raft_consensus.cc:1261] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 1 FOLLOWER]: Refusing update from remote peer b9064d4591d7499bb26d9941e28a7ab0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:38.220345 16546 consensus_queue.cc:1035] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:38.220472 16515 consensus_queue.cc:1035] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:38.220959 16514 tablet_replica.cc:439] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2: TxnStatusTablet state changed. Reason: New leader b9064d4591d7499bb26d9941e28a7ab0. Latest consensus state: current_term: 1 leader_uuid: "b9064d4591d7499bb26d9941e28a7ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } }
I20260806 13:06:38.221055 16514 tablet_replica.cc:442] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:06:38.221159 16518 tablet_replica.cc:439] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371: TxnStatusTablet state changed. Reason: New leader b9064d4591d7499bb26d9941e28a7ab0. Latest consensus state: current_term: 1 leader_uuid: "b9064d4591d7499bb26d9941e28a7ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } }
I20260806 13:06:38.221238 16518 tablet_replica.cc:442] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:06:38.221743 16546 tablet_replica.cc:439] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9064d4591d7499bb26d9941e28a7ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } }
I20260806 13:06:38.221820 16546 tablet_replica.cc:442] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0: This TxnStatusTablet replica's current role is: LEADER
I20260806 13:06:38.222199 16515 tablet_replica.cc:439] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9064d4591d7499bb26d9941e28a7ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } }
I20260806 13:06:38.222282 16515 tablet_replica.cc:442] T 0ffcfa711f324e998cabc6986adbc40f P b9064d4591d7499bb26d9941e28a7ab0: This TxnStatusTablet replica's current role is: LEADER
I20260806 13:06:38.222836 16514 tablet_replica.cc:439] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9064d4591d7499bb26d9941e28a7ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } }
I20260806 13:06:38.222921 16514 tablet_replica.cc:442] T 0ffcfa711f324e998cabc6986adbc40f P 7bd6a65095bc467ebe16c0cc8d6dc7d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:06:38.223287 16518 tablet_replica.cc:439] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9064d4591d7499bb26d9941e28a7ab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 } } peers { permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 } } peers { permanent_uuid: "b9064d4591d7499bb26d9941e28a7ab0" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 34307 } } }
I20260806 13:06:38.223362 16518 tablet_replica.cc:442] T 0ffcfa711f324e998cabc6986adbc40f P 23c96c2328394a0986b8654b9b458371: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:06:38.226377 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:38.224147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45552 (local address 127.13.224.67:34307)
0806 13:06:38.224189 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:38.224191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:38.224222 (+ 31us) server_negotiation.cc:407] Connection header received
0806 13:06:38.224295 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:38.224296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:38.224306 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:38.224355 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:06:38.224449 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.224914 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.225610 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.225808 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.225860 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:38.226033 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:06:38.226065 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:06:38.226196 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:38.226286 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 13:06:38.226309 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:06:38.231184 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:38.229023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:55557 (local address 127.13.224.67:34307)
0806 13:06:38.229067 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:38.229069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:38.229076 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:06:38.229123 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:38.229123 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:38.229131 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:38.229175 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:06:38.229303 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.229723 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.230543 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.230755 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.230809 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:38.230969 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:06:38.230998 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:06:38.231037 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:38.231099 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 13:06:38.231113 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 13:06:38.231230 16479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:38.228953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:34307 (local address 127.13.224.67:55557)
0806 13:06:38.229026 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:38.229035 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:38.229080 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:38.229183 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:38.229184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:38.229185 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 13:06:38.229247 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:38.229249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.229805 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.229806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:38.230470 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:38.230473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.230824 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.230825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:38.230885 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:38.230899 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 13:06:38.231049 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 13:06:38.231050 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 13:06:38.231096 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 13:06:38.231131 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 13:06:38.233654 16012 raft_consensus.cc:1261] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 1 FOLLOWER]: Refusing update from remote peer b9064d4591d7499bb26d9941e28a7ab0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:38.233842 16207 raft_consensus.cc:1261] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371 [term 1 FOLLOWER]: Refusing update from remote peer b9064d4591d7499bb26d9941e28a7ab0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:38.233834 16546 consensus_queue.cc:1035] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bd6a65095bc467ebe16c0cc8d6dc7d2" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:38.234043 16546 consensus_queue.cc:1035] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23c96c2328394a0986b8654b9b458371" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 46103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:38.255980 15774 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:38.256089 15774 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:38.256712 15774 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c82cc65a483e408cbb38c307e139d558: Sending DeleteTablet for 3 replicas of tablet 92621bd8608943d69f7b8a40d901c1fa
I20260806 13:06:38.256939 16174 tablet_service.cc:1514] Processing DeleteTablet for tablet 92621bd8608943d69f7b8a40d901c1fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:38 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:41870
I20260806 13:06:38.257058 16537 tablet_replica.cc:330] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371: stopping tablet replica
I20260806 13:06:38.257072 16364 tablet_service.cc:1514] Processing DeleteTablet for tablet 92621bd8608943d69f7b8a40d901c1fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:38 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:48728
I20260806 13:06:38.257068 15971 tablet_service.cc:1514] Processing DeleteTablet for tablet 92621bd8608943d69f7b8a40d901c1fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:38 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:40202
I20260806 13:06:38.257131 16537 raft_consensus.cc:2229] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:38.257150 16536 tablet_replica.cc:330] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0: stopping tablet replica
I20260806 13:06:38.257161 16539 tablet_replica.cc:330] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2: stopping tablet replica
I20260806 13:06:38.257169 16537 raft_consensus.cc:2258] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:38.257193 16536 raft_consensus.cc:2229] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:06:38.257203 16539 raft_consensus.cc:2229] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:38.257234 16539 raft_consensus.cc:2258] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:38.257272 16536 raft_consensus.cc:2258] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:38.257586 16536 ts_tablet_manager.cc:1905] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:38.257591 16539 ts_tablet_manager.cc:1905] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:38.259155 16536 ts_tablet_manager.cc:1918] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:06:38.259214 16536 log.cc:1196] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/92621bd8608943d69f7b8a40d901c1fa
I20260806 13:06:38.259310 16539 ts_tablet_manager.cc:1918] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:06:38.259371 16539 log.cc:1196] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-0/wal/wals/92621bd8608943d69f7b8a40d901c1fa
I20260806 13:06:38.259424 16536 ts_tablet_manager.cc:1939] T 92621bd8608943d69f7b8a40d901c1fa P b9064d4591d7499bb26d9941e28a7ab0: Deleting consensus metadata
I20260806 13:06:38.259593 16539 ts_tablet_manager.cc:1939] T 92621bd8608943d69f7b8a40d901c1fa P 7bd6a65095bc467ebe16c0cc8d6dc7d2: Deleting consensus metadata
I20260806 13:06:38.259809 15757 catalog_manager.cc:4841] TS b9064d4591d7499bb26d9941e28a7ab0 (127.13.224.67:34307): tablet 92621bd8608943d69f7b8a40d901c1fa (table test-table [id=557399158f0c44f989ffc6c81a8b0fe1]) successfully deleted
I20260806 13:06:38.259979 15756 catalog_manager.cc:4841] TS 7bd6a65095bc467ebe16c0cc8d6dc7d2 (127.13.224.65:43927): tablet 92621bd8608943d69f7b8a40d901c1fa (table test-table [id=557399158f0c44f989ffc6c81a8b0fe1]) successfully deleted
I20260806 13:06:38.260200 16537 ts_tablet_manager.cc:1905] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:38.261693 16537 ts_tablet_manager.cc:1918] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:06:38.261756 16537 log.cc:1196] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/92621bd8608943d69f7b8a40d901c1fa
I20260806 13:06:38.261950 16537 ts_tablet_manager.cc:1939] T 92621bd8608943d69f7b8a40d901c1fa P 23c96c2328394a0986b8654b9b458371: Deleting consensus metadata
I20260806 13:06:38.262295 15755 catalog_manager.cc:4841] TS 23c96c2328394a0986b8654b9b458371 (127.13.224.66:46103): tablet 92621bd8608943d69f7b8a40d901c1fa (table test-table [id=557399158f0c44f989ffc6c81a8b0fe1]) successfully deleted
Aug 06 13:06:38 dist-test-slave-q9h9 krb5kdc[15714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021597, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.224.65@KRBTEST.COM
I20260806 13:06:38.273172 16504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:38.264306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35142 (local address 127.13.224.65:43927)
0806 13:06:38.264375 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:38.264377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:38.264385 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:06:38.264406 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:38.264407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:38.264417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:38.264461 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:38.265183 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.265637 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.266727 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.266919 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.266968 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:38.270203 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:38.270213 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:38.270214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:38.270220 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:38.271976 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:38.272328 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:38.272332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:38.272332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:38.272400 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:38.272625 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:38.272626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:38.272626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:38.272769 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:38.272899 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:38.272992 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 13:06:38.273071 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 13:06:38.273447 16060 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35142
I20260806 13:06:38.280902 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:38.274527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.13.224.126:45169)
0806 13:06:38.274581 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:38.274583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:38.274651 (+ 68us) server_negotiation.cc:407] Connection header received
0806 13:06:38.274719 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:38.274719 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:38.274727 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:38.274769 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:38.275370 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.275859 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.276874 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:38.277087 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:38.277151 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:38.278531 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:38.278533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:38.278533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:38.278540 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:38.279757 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:38.280150 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:38.280151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:38.280152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:38.280209 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:38.280416 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:38.280417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:38.280417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:38.280513 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:38.280585 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:38.280774 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 13:06:38.280834 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 13:06:38.281145 15774 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34866
I20260806 13:06:38.281876 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 15881
I20260806 13:06:38.302410 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 16082
I20260806 13:06:38.314917 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 16279
I20260806 13:06:38.326828 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 15729
2026-08-06T13:06:38Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2402 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:06:38 dist-test-slave-q9h9 krb5kdc[16581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:06:38 dist-test-slave-q9h9 krb5kdc[16581](info): set up 2 sockets
Aug 06 13:06:38 dist-test-slave-q9h9 krb5kdc[16581](info): commencing operation
krb5kdc: starting...
W20260806 13:06:39.434954 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:06:39 dist-test-slave-q9h9 krb5kdc[16581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:06:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:06:39.542361 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:42265
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:35837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:42265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:39.623522 16603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:39.623728 16603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:39.623760 16603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:39.624881 16603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:39.624927 16603 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 13:06:39.624950 16603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:39.624971 16603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:39.624991 16603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:39.625015 16603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:06:39.627111 16603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:35837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:42265
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:42265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:39.627374 16603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:39.627599 16603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:39.633306 16615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:39.633322 16618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:39.633312 16614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:39.633371 16603 server_base.cc:1034] running on GCE node
I20260806 13:06:39.633757 16603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:39.634089 16603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:39.635241 16603 hybrid_clock.cc:648] HybridClock initialized: now 1786021599635230 us; error 30 us; skew 500 ppm
Aug 06 13:06:39 dist-test-slave-q9h9 krb5kdc[16581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021599, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:39.637687 16603 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:06:39.638230 16603 webserver.cc:460] Webserver started at http://127.13.224.126:40865/ using document root <none> and password file <none>
I20260806 13:06:39.638481 16603 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:39.638531 16603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:39.638679 16603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:39.639721 16603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "3fb4104091384da49e66ac27dd40c2f6"
format_stamp: "Formatted at 2026-08-06 13:06:39 on dist-test-slave-q9h9"
server_key: "93fd715386ccf5579d22b42504aff423"
server_key_iv: "7a23baa25a2b1c1bbb052481365fa835"
server_key_version: "encryptionkey@0"
I20260806 13:06:39.640077 16603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "3fb4104091384da49e66ac27dd40c2f6"
format_stamp: "Formatted at 2026-08-06 13:06:39 on dist-test-slave-q9h9"
server_key: "93fd715386ccf5579d22b42504aff423"
server_key_iv: "7a23baa25a2b1c1bbb052481365fa835"
server_key_version: "encryptionkey@0"
I20260806 13:06:39.641417 16603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:06:39.642148 16627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:39.642318 16603 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:39.642419 16603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "3fb4104091384da49e66ac27dd40c2f6"
format_stamp: "Formatted at 2026-08-06 13:06:39 on dist-test-slave-q9h9"
server_key: "93fd715386ccf5579d22b42504aff423"
server_key_iv: "7a23baa25a2b1c1bbb052481365fa835"
server_key_version: "encryptionkey@0"
I20260806 13:06:39.642490 16603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:39.666960 16603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:39.668721 16603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:39.668872 16603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:39.673624 16603 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:42265
I20260806 13:06:39.673664 16726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:42265 every 8 connection(s)
I20260806 13:06:39.674001 16603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:39.674656 16727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:39.676584 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 16603
I20260806 13:06:39.676676 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:39.676851 14209 external_mini_cluster.cc:1428] Setting key b9d75b79ace6df7db7089e0f2e85de09
I20260806 13:06:39.677080 16727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6: Bootstrap starting.
I20260806 13:06:39.677758 16727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:39.678056 16727 log.cc:824] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:39.678706 16727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6: No bootstrap required, opened a new log
I20260806 13:06:39.680454 16727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb4104091384da49e66ac27dd40c2f6" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42265 } }
I20260806 13:06:39.680598 16727 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:39.680631 16727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb4104091384da49e66ac27dd40c2f6, State: Initialized, Role: FOLLOWER
I20260806 13:06:39.680761 16727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb4104091384da49e66ac27dd40c2f6" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42265 } }
I20260806 13:06:39.680833 16727 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:39.680864 16727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:39.680910 16727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:39.681387 16727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb4104091384da49e66ac27dd40c2f6" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42265 } }
I20260806 13:06:39.681480 16727 leader_election.cc:302] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fb4104091384da49e66ac27dd40c2f6; no voters:
I20260806 13:06:39.681632 16727 leader_election.cc:288] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:39.681656 16733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:39.681766 16733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb4104091384da49e66ac27dd40c2f6, State: Running, Role: LEADER
I20260806 13:06:39.681846 16733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb4104091384da49e66ac27dd40c2f6" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42265 } }
I20260806 13:06:39.681885 16727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:39.682157 16733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fb4104091384da49e66ac27dd40c2f6. Latest consensus state: current_term: 1 leader_uuid: "3fb4104091384da49e66ac27dd40c2f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb4104091384da49e66ac27dd40c2f6" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42265 } } }
I20260806 13:06:39.682170 16735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fb4104091384da49e66ac27dd40c2f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb4104091384da49e66ac27dd40c2f6" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42265 } } }
I20260806 13:06:39.682230 16733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:39.682396 16735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6 [sys.catalog]: This master's current role is: LEADER
W20260806 13:06:39.683629 16753 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:06:39.683682 16753 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 06 13:06:39 dist-test-slave-q9h9 krb5kdc[16581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:39.683728 16740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:39.683923 16740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:39.685415 16740 catalog_manager.cc:1347] Generated new cluster ID: 9f2d73c23d2c4d19b4e3e601ac0a7c3e
I20260806 13:06:39.685467 16740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:06:39.687706 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:39.677449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58814 (local address 127.13.224.126:42265)
0806 13:06:39.677630 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:39.677640 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:39.677670 (+ 30us) server_negotiation.cc:407] Connection header received
0806 13:06:39.678219 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:39.678229 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:39.678286 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:39.678423 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:39.679150 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.680024 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.680836 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.681321 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.681430 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:39.684216 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:39.684220 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:39.684220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:39.684231 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:39.686146 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:39.686547 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:39.686549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:39.686550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:39.686646 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:39.686864 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:39.686865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:39.686865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:39.687127 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:39.687267 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:39.687401 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 13:06:39.687522 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
I20260806 13:06:39.691515 16740 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:39.691974 16740 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:39.701028 16740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6: Generated new TSK 0
I20260806 13:06:39.701165 16740 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:06:39.728070 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:42265
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:35837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:39.807729 16761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:39.807925 16761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:39.807948 16761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:39.809070 16761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:39.809109 16761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:39.809128 16761 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 13:06:39.809139 16761 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 13:06:39.809154 16761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:06:39.810659 16761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:35837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.224.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:39.810892 16761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:39.811105 16761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:39.815933 16768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:39.815948 16769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:39.815948 16771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:39.816169 16761 server_base.cc:1034] running on GCE node
I20260806 13:06:39.816469 16761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:39.816679 16761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:39.817811 16761 hybrid_clock.cc:648] HybridClock initialized: now 1786021599817802 us; error 33 us; skew 500 ppm
Aug 06 13:06:39 dist-test-slave-q9h9 krb5kdc[16581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021599, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:39.820139 16761 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:06:39.820695 16761 webserver.cc:460] Webserver started at http://127.13.224.65:38575/ using document root <none> and password file <none>
I20260806 13:06:39.820955 16761 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:39.821015 16761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:39.821143 16761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:39.822196 16761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "37eb8641c2224cb490aed8424f8e9a22"
format_stamp: "Formatted at 2026-08-06 13:06:39 on dist-test-slave-q9h9"
server_key: "6b9508c2df991a8d5a4fb56f207f8b11"
server_key_iv: "4f1c2d28dec9157e327079f5b93eb8d1"
server_key_version: "encryptionkey@0"
I20260806 13:06:39.822525 16761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "37eb8641c2224cb490aed8424f8e9a22"
format_stamp: "Formatted at 2026-08-06 13:06:39 on dist-test-slave-q9h9"
server_key: "6b9508c2df991a8d5a4fb56f207f8b11"
server_key_iv: "4f1c2d28dec9157e327079f5b93eb8d1"
server_key_version: "encryptionkey@0"
I20260806 13:06:39.823738 16761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:06:39.824376 16784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:39.824539 16761 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:39.824620 16761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "37eb8641c2224cb490aed8424f8e9a22"
format_stamp: "Formatted at 2026-08-06 13:06:39 on dist-test-slave-q9h9"
server_key: "6b9508c2df991a8d5a4fb56f207f8b11"
server_key_iv: "4f1c2d28dec9157e327079f5b93eb8d1"
server_key_version: "encryptionkey@0"
I20260806 13:06:39.824680 16761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:39.831761 16761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:39.833345 16761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:39.833472 16761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:39.833917 16761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:39.833958 16761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:39.833987 16761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:39.834002 16761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 13:06:39 dist-test-slave-q9h9 krb5kdc[16581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021599, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:39.840350 16761 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:40223
I20260806 13:06:39.840416 16944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:40223 every 8 connection(s)
I20260806 13:06:39.840693 16761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:06:39.842191 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 16761
I20260806 13:06:39.842286 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:06:39.842476 14209 external_mini_cluster.cc:1428] Setting key 41bf22e8f5b330a770659f450a55a13b
I20260806 13:06:39.843627 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:39.834410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:49127 (local address 127.13.224.126:42265)
0806 13:06:39.834480 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:39.834483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:39.834620 (+ 137us) server_negotiation.cc:407] Connection header received
0806 13:06:39.835069 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:39.835071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:39.835085 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:39.835155 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:39.835876 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.836592 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.838048 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.838363 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.838438 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:39.840866 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:39.840868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:39.840868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:39.840877 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:39.842203 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:39.842643 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:39.842645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:39.842646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:39.842714 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:39.842972 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:39.842973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:39.842973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:39.843101 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:39.843170 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:39.843494 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 13:06:39.843564 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 13:06:39.843818 16797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:39.834343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:42265 (local address 127.13.224.65:49127)
0806 13:06:39.834496 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:39.834532 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:39.834972 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:39.835213 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:39.835216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:39.835494 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:39.835768 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:39.835774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.836710 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.836711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:39.837940 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:39.837943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.838465 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.838466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:39.838580 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:39.839054 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:39.839071 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:39.840782 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:39.842282 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:39.842290 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:39.842292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:39.842564 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:39.842790 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:39.842791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:39.842791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:39.842910 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:39.843193 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:39.843200 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:39.843363 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 13:06:39.843474 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 13:06:39.843617 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260806 13:06:39.844313 16946 heartbeater.cc:347] Connected to a master server at 127.13.224.126:42265
I20260806 13:06:39.844396 16946 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:39.844568 16946 heartbeater.cc:511] Master 127.13.224.126:42265 requested a full tablet report, sending...
I20260806 13:06:39.845117 16651 ts_manager.cc:194] Registered new tserver with Master: 37eb8641c2224cb490aed8424f8e9a22 (127.13.224.65:40223)
I20260806 13:06:39.845989 16651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:49127
I20260806 13:06:39.850332 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:39.845283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:34683 (local address 127.13.224.126:42265)
0806 13:06:39.845323 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:39.845325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:39.845467 (+ 142us) server_negotiation.cc:407] Connection header received
0806 13:06:39.845502 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:39.845503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:39.845512 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:39.845552 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:39.845833 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.846424 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.847106 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.847298 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.847351 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:39.848178 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:39.848179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:39.848180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:39.848186 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:39.849339 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:39.849638 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:39.849639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:39.849639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:39.849690 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:39.849863 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:39.849863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:39.849864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:39.849961 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:39.850034 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:39.850182 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 13:06:39.850246 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:06:39.850339 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:39.845255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:42265 (local address 127.13.224.65:34683)
0806 13:06:39.845398 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:39.845406 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:39.845456 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:39.845556 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:39.845556 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:39.845716 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:39.845763 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:39.845764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.846513 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.846514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:39.847028 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:39.847029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.847363 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.847364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:39.847428 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:39.847852 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:39.847855 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:39.848096 (+ 241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:39.849416 (+ 1320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:39.849417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:39.849418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:39.849568 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:39.849748 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:39.849749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:39.849749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:39.849805 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:39.850039 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:39.850040 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:39.850086 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 13:06:39.850145 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 13:06:39.850238 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:06:39.866215 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:42265
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:35837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:39.946218 16959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:39.946408 16959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:39.946431 16959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:39.947579 16959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:39.947619 16959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:39.947639 16959 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 13:06:39.947661 16959 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 13:06:39.947683 16959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:06:39.949146 16959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:35837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.224.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:39.949345 16959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:39.949553 16959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:39.954288 16971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:39.954336 16967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:39.954308 16959 server_base.cc:1034] running on GCE node
W20260806 13:06:39.954293 16966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:39.954663 16959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:39.954900 16959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:39.956037 16959 hybrid_clock.cc:648] HybridClock initialized: now 1786021599956021 us; error 32 us; skew 500 ppm
Aug 06 13:06:39 dist-test-slave-q9h9 krb5kdc[16581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021599, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:39.958314 16959 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:06:39.958673 16959 webserver.cc:460] Webserver started at http://127.13.224.66:35813/ using document root <none> and password file <none>
I20260806 13:06:39.958845 16959 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:39.958889 16959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:39.958973 16959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:39.959896 16959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "9060a84263164dfcb4fe1f0b13cc4d7e"
format_stamp: "Formatted at 2026-08-06 13:06:39 on dist-test-slave-q9h9"
server_key: "f919bbf2b0240da9a2c20f708006b91c"
server_key_iv: "ce92582f0a8c43d592f0f2407802d9ba"
server_key_version: "encryptionkey@0"
I20260806 13:06:39.960168 16959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "9060a84263164dfcb4fe1f0b13cc4d7e"
format_stamp: "Formatted at 2026-08-06 13:06:39 on dist-test-slave-q9h9"
server_key: "f919bbf2b0240da9a2c20f708006b91c"
server_key_iv: "ce92582f0a8c43d592f0f2407802d9ba"
server_key_version: "encryptionkey@0"
I20260806 13:06:39.961386 16959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:06:39.962045 16980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:39.962191 16959 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:39.962265 16959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "9060a84263164dfcb4fe1f0b13cc4d7e"
format_stamp: "Formatted at 2026-08-06 13:06:39 on dist-test-slave-q9h9"
server_key: "f919bbf2b0240da9a2c20f708006b91c"
server_key_iv: "ce92582f0a8c43d592f0f2407802d9ba"
server_key_version: "encryptionkey@0"
I20260806 13:06:39.962333 16959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:39.972316 16959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:39.973871 16959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:39.973987 16959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:39.974470 16959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:39.974509 16959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:39.974534 16959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:39.974550 16959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 13:06:39 dist-test-slave-q9h9 krb5kdc[16581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021599, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:39.980911 16959 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:42751
I20260806 13:06:39.980978 17161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:42751 every 8 connection(s)
I20260806 13:06:39.981259 16959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:06:39.983537 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:39.974903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:43231 (local address 127.13.224.126:42265)
0806 13:06:39.974953 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:39.974955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:39.975154 (+ 199us) server_negotiation.cc:407] Connection header received
0806 13:06:39.975598 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:39.975600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:39.975613 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:39.975664 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:39.976359 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.977111 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.978454 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.978731 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.978794 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:39.981078 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:39.981080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:39.981080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:39.981088 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:39.982440 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:39.982813 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:39.982814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:39.982815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:39.982866 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:39.983070 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:39.983070 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:39.983070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:39.983159 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:39.983202 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:39.983416 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 13:06:39.983475 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 13:06:39.983695 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:39.974898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:42265 (local address 127.13.224.66:43231)
0806 13:06:39.975041 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:39.975075 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:39.975495 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:39.975732 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:39.975744 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:39.976018 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:39.976274 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:39.976280 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.977196 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.977198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:39.978364 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:39.978367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.978821 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.978822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:39.978941 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:39.979397 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:39.979405 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:39.980997 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:39.982536 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:39.982542 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:39.982545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:39.982751 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:39.982908 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:39.982908 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:39.982908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:39.983025 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:39.983199 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:39.983204 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:39.983317 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 13:06:39.983395 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 13:06:39.983504 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260806 13:06:39.984153 17162 heartbeater.cc:347] Connected to a master server at 127.13.224.126:42265
I20260806 13:06:39.984231 17162 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:39.984408 17162 heartbeater.cc:511] Master 127.13.224.126:42265 requested a full tablet report, sending...
I20260806 13:06:39.984805 16654 ts_manager.cc:194] Registered new tserver with Master: 9060a84263164dfcb4fe1f0b13cc4d7e (127.13.224.66:42751)
I20260806 13:06:39.985258 16654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:43231
I20260806 13:06:39.989753 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:39.985090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:44839 (local address 127.13.224.126:42265)
0806 13:06:39.985127 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:39.985128 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:39.985266 (+ 138us) server_negotiation.cc:407] Connection header received
0806 13:06:39.985290 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:39.985291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:39.985300 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:39.985351 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:39.985724 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.986305 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.986926 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.987090 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.987131 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:39.987813 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:39.987814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:39.987814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:39.987820 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:39.989004 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:39.989253 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:39.989254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:39.989254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:39.989293 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:39.989398 (+ 105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:39.989398 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:39.989398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:39.989508 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:39.989551 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:39.989648 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 13:06:39.989698 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:06:39.989761 17168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:39.985037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:42265 (local address 127.13.224.66:44839)
0806 13:06:39.985191 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:39.985199 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:39.985254 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:39.985394 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:39.985395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:39.985589 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:39.985647 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:39.985649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.986385 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.986386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:39.986864 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:39.986865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:39.987140 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:39.987140 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:39.987189 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:39.987504 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:39.987508 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:39.987745 (+ 237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:39.989065 (+ 1320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:39.989065 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:39.989065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:39.989200 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:39.989325 (+ 125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:39.989326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:39.989326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:39.989366 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:39.989553 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:39.989553 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:39.989586 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 13:06:39.989628 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 13:06:39.989691 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":65,"threads_started":1}
I20260806 13:06:39.990099 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 16959
I20260806 13:06:39.990201 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:06:39.990383 14209 external_mini_cluster.cc:1428] Setting key d33391d89a0e278388e8255aaa2c9336
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:06:40.012348 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:42265
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:35837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:40.092442 17173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:40.092640 17173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:40.092665 17173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:40.093797 17173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:40.093845 17173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:40.093869 17173 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 13:06:40.093896 17173 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 13:06:40.093930 17173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:06:40.095458 17173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:35837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.224.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:40.095686 17173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:40.095907 17173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:40.100701 17179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:40.100728 17181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:40.100878 17173 server_base.cc:1034] running on GCE node
W20260806 13:06:40.100728 17186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:40.101120 17173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:40.101320 17173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:40.102452 17173 hybrid_clock.cc:648] HybridClock initialized: now 1786021600102443 us; error 33 us; skew 500 ppm
Aug 06 13:06:40 dist-test-slave-q9h9 krb5kdc[16581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021600, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:40.104792 17173 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:06:40.105248 17173 webserver.cc:460] Webserver started at http://127.13.224.67:42247/ using document root <none> and password file <none>
I20260806 13:06:40.105437 17173 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:40.105480 17173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:40.105593 17173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:40.106630 17173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "5691b87b7f43439e83884208db071f32"
format_stamp: "Formatted at 2026-08-06 13:06:40 on dist-test-slave-q9h9"
server_key: "82a19a63fd4e77c4f1e31a500d04d403"
server_key_iv: "7431bc4ca59c4ff19ba81dc3b1ad832c"
server_key_version: "encryptionkey@0"
I20260806 13:06:40.106949 17173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "5691b87b7f43439e83884208db071f32"
format_stamp: "Formatted at 2026-08-06 13:06:40 on dist-test-slave-q9h9"
server_key: "82a19a63fd4e77c4f1e31a500d04d403"
server_key_iv: "7431bc4ca59c4ff19ba81dc3b1ad832c"
server_key_version: "encryptionkey@0"
I20260806 13:06:40.108124 17173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:06:40.108781 17197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:40.108939 17173 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:40.109014 17173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "5691b87b7f43439e83884208db071f32"
format_stamp: "Formatted at 2026-08-06 13:06:40 on dist-test-slave-q9h9"
server_key: "82a19a63fd4e77c4f1e31a500d04d403"
server_key_iv: "7431bc4ca59c4ff19ba81dc3b1ad832c"
server_key_version: "encryptionkey@0"
I20260806 13:06:40.109077 17173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:40.128592 17173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:40.130249 17173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:40.130369 17173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:40.130847 17173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:40.130890 17173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:40.130911 17173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:40.130923 17173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 13:06:40 dist-test-slave-q9h9 krb5kdc[16581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021600, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:40.137637 17173 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:45991
I20260806 13:06:40.137718 17373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:45991 every 8 connection(s)
I20260806 13:06:40.138065 17173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:06:40.140157 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.131281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:35987 (local address 127.13.224.126:42265)
0806 13:06:40.131348 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:40.131351 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:40.131522 (+ 171us) server_negotiation.cc:407] Connection header received
0806 13:06:40.131978 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:40.131980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:40.131993 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:40.132066 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:40.132816 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.133541 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.135030 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.135328 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.135410 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.137801 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:40.137803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:40.137803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:40.137812 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:40.139154 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:40.139505 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:40.139505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:40.139506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:40.139552 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:40.139710 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:40.139711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:40.139711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:40.139801 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:40.139845 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:40.140037 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 13:06:40.140096 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 13:06:40.140343 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.131213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:42265 (local address 127.13.224.67:35987)
0806 13:06:40.131399 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:40.131438 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:40.131882 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:40.132143 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:40.132148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:40.132439 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:40.132699 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.132705 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.133636 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.133637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.134933 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.134936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.135434 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.135435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.135574 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.136027 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:40.136035 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:40.137691 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:40.139227 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:40.139239 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:40.139241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:40.139443 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:40.139589 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:40.139589 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:40.139589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:40.139668 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:40.139849 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:40.139851 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:40.139951 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 13:06:40.140026 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 13:06:40.140135 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260806 13:06:40.140771 17374 heartbeater.cc:347] Connected to a master server at 127.13.224.126:42265
I20260806 13:06:40.140853 17374 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:40.141028 17374 heartbeater.cc:511] Master 127.13.224.126:42265 requested a full tablet report, sending...
I20260806 13:06:40.141376 16651 ts_manager.cc:194] Registered new tserver with Master: 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991)
I20260806 13:06:40.141812 16651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:35987
I20260806 13:06:40.146450 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 17173
I20260806 13:06:40.146549 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:06:40.146739 14209 external_mini_cluster.cc:1428] Setting key a88bb049d7645deedbc9307a272efe29
I20260806 13:06:40.146886 17382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.141740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:42265 (local address 127.13.224.67:48527)
0806 13:06:40.141882 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:40.141891 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:40.141937 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:40.142291 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:40.142291 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:40.142464 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:40.142515 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.142517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.143118 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.143119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.143605 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.143606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.143963 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.143964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.144038 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.144351 (+ 313us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:40.144355 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:40.144618 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:40.145954 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:40.145955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:40.145956 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:40.146143 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:40.146329 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:40.146330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:40.146331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:40.146384 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:40.146624 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:40.146625 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:40.146662 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 13:06:40.146717 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 13:06:40.146798 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260806 13:06:40.146924 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.141812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:48527 (local address 127.13.224.126:42265)
0806 13:06:40.141944 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:40.141945 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:40.141953 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:06:40.141981 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:40.141982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:40.141990 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:40.142050 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:40.142585 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.143026 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.143681 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.143898 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.143948 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.144692 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:40.144694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:40.144694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:40.144700 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:40.145884 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:40.146202 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:40.146203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:40.146203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:40.146251 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:40.146442 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:40.146443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:40.146443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:40.146565 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:40.146631 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:40.146781 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 13:06:40.146850 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260806 13:06:40.147660 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 13:06:40 dist-test-slave-q9h9 krb5kdc[16581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 13:06:40 dist-test-slave-q9h9 krb5kdc[16581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:40.169745 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.163807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.13.224.126:42265)
0806 13:06:40.163859 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:40.163861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:40.163870 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:06:40.163892 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:40.163893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:40.163900 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:40.163946 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:40.164418 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.164909 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.165563 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.165740 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.165795 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.167771 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:40.167773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:40.167773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:40.167780 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:40.168776 (+ 996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:40.169090 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:40.169091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:40.169092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:40.169139 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:40.169310 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:40.169311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:40.169312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:40.169406 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:40.169492 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:40.169595 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 13:06:40.169664 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 13:06:40.171021 16654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:06:40.172575 16654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:06:40.178597 17398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.174792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41522 (local address 127.13.224.66:42751)
0806 13:06:40.174952 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:40.174956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:40.174967 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:06:40.175004 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:40.175006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:40.175022 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:40.175114 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:40.175280 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.176387 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.177593 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.178194 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.178266 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.178327 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:40.178462 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 13:06:40.178501 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260806 13:06:40.178630 17395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.174605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:42751 (local address 127.0.0.1:41522)
0806 13:06:40.174755 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:40.174782 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:40.174861 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:40.175119 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:40.175122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:40.175124 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:40.175191 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.175193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.176470 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.176471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.177508 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.177510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.178275 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.178276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.178353 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.178366 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 13:06:40.178432 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 13:06:40.178484 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":9,"thread_start_us":55,"threads_started":1}
I20260806 13:06:40.178669 17399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.174861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.13.224.67:45991)
0806 13:06:40.175028 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:40.175032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:40.175258 (+ 226us) server_negotiation.cc:407] Connection header received
0806 13:06:40.175367 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:40.175370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:40.175384 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:40.175432 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:40.175619 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.176613 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.177536 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.178229 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.178293 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.178344 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:40.178533 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 13:06:40.178570 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260806 13:06:40.178860 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.174649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:45991 (local address 127.0.0.1:42292)
0806 13:06:40.175194 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:40.175203 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:40.175286 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:40.175506 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:40.175507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:40.175509 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:40.175546 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.175547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.176682 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.176683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.177470 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.177472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.178378 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.178379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.178439 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.178444 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:06:40.178747 (+ 303us) client_negotiation.cc:240] Negotiation successful
0806 13:06:40.178796 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":76,"threads_started":1}
I20260806 13:06:40.179015 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.174606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:40223 (local address 127.0.0.1:47422)
0806 13:06:40.174955 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:40.174964 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:40.175001 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:40.175328 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:40.175328 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:40.175330 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:40.175373 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.175374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.176285 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.176286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.177696 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.177698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.178807 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.178808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.178868 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.178881 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 13:06:40.178932 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 13:06:40.178966 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":52,"threads_started":1}
I20260806 13:06:40.179235 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.175007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47422 (local address 127.13.224.65:40223)
0806 13:06:40.175191 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:40.175195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:40.175205 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:06:40.175242 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:40.175243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:40.175258 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:40.175322 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:40.175442 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.176199 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.177773 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.178511 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.178568 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.178620 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:40.179098 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 13:06:40.179142 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260806 13:06:40.179507 17050 tablet_service.cc:1467] Processing CreateTablet for tablet 826c5f5bb7f54472b16700e53d0014b0 (DEFAULT_TABLE table=test-table [id=a2bd9e76171940fea1de6c493ba70a02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:40.179525 17274 tablet_service.cc:1467] Processing CreateTablet for tablet 826c5f5bb7f54472b16700e53d0014b0 (DEFAULT_TABLE table=test-table [id=a2bd9e76171940fea1de6c493ba70a02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:40.179859 17050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 826c5f5bb7f54472b16700e53d0014b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:40.179879 17274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 826c5f5bb7f54472b16700e53d0014b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:40.179806 16855 tablet_service.cc:1467] Processing CreateTablet for tablet 826c5f5bb7f54472b16700e53d0014b0 (DEFAULT_TABLE table=test-table [id=a2bd9e76171940fea1de6c493ba70a02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:40.180099 16855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 826c5f5bb7f54472b16700e53d0014b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:40.183105 17406 tablet_bootstrap.cc:492] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32: Bootstrap starting.
I20260806 13:06:40.183710 17407 tablet_bootstrap.cc:492] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22: Bootstrap starting.
I20260806 13:06:40.183952 17406 tablet_bootstrap.cc:654] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:40.183986 17404 tablet_bootstrap.cc:492] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e: Bootstrap starting.
I20260806 13:06:40.184259 17406 log.cc:824] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:40.184361 17407 tablet_bootstrap.cc:654] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:40.184600 17404 tablet_bootstrap.cc:654] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:40.184681 17407 log.cc:824] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:40.184960 17406 tablet_bootstrap.cc:492] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32: No bootstrap required, opened a new log
I20260806 13:06:40.184950 17404 log.cc:824] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:40.185034 17406 ts_tablet_manager.cc:1397] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:06:40.185570 17407 tablet_bootstrap.cc:492] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22: No bootstrap required, opened a new log
I20260806 13:06:40.185623 17407 ts_tablet_manager.cc:1397] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:06:40.186739 17406 raft_consensus.cc:348] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } }
I20260806 13:06:40.186864 17406 raft_consensus.cc:374] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:40.186896 17406 raft_consensus.cc:729] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5691b87b7f43439e83884208db071f32, State: Initialized, Role: FOLLOWER
I20260806 13:06:40.187008 17406 consensus_queue.cc:260] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } }
I20260806 13:06:40.186981 17407 raft_consensus.cc:348] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } }
I20260806 13:06:40.187117 17407 raft_consensus.cc:374] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:40.187146 17407 raft_consensus.cc:729] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37eb8641c2224cb490aed8424f8e9a22, State: Initialized, Role: FOLLOWER
I20260806 13:06:40.187266 17406 ts_tablet_manager.cc:1428] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 13:06:40.187233 17407 consensus_queue.cc:260] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } }
I20260806 13:06:40.187311 17374 heartbeater.cc:503] Master 127.13.224.126:42265 was elected leader, sending a full tablet report...
I20260806 13:06:40.187484 17407 ts_tablet_manager.cc:1428] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:06:40.187638 16946 heartbeater.cc:503] Master 127.13.224.126:42265 was elected leader, sending a full tablet report...
I20260806 13:06:40.187887 17404 tablet_bootstrap.cc:492] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e: No bootstrap required, opened a new log
I20260806 13:06:40.187944 17404 ts_tablet_manager.cc:1397] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 13:06:40.189469 17404 raft_consensus.cc:348] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } }
I20260806 13:06:40.189617 17404 raft_consensus.cc:374] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:40.189659 17404 raft_consensus.cc:729] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9060a84263164dfcb4fe1f0b13cc4d7e, State: Initialized, Role: FOLLOWER
I20260806 13:06:40.189761 17404 consensus_queue.cc:260] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } }
I20260806 13:06:40.189978 17404 ts_tablet_manager.cc:1428] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:06:40.190075 17162 heartbeater.cc:503] Master 127.13.224.126:42265 was elected leader, sending a full tablet report...
W20260806 13:06:40.232029 17163 tablet.cc:2369] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:06:40.339413 17416 raft_consensus.cc:482] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:06:40.339534 17416 raft_consensus.cc:504] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } }
I20260806 13:06:40.339918 17416 leader_election.cc:288] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991), 37eb8641c2224cb490aed8424f8e9a22 (127.13.224.65:40223)
W20260806 13:06:40.341367 16947 tablet.cc:2369] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:06:40.342857 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.340155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:55387 (local address 127.13.224.65:40223)
0806 13:06:40.340201 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:40.340203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:40.340349 (+ 146us) server_negotiation.cc:407] Connection header received
0806 13:06:40.340402 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:40.340403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:40.340413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:40.340491 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:40.340652 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.341137 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.342149 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.342528 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.342583 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.342590 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:40.342753 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 13:06:40.342796 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:06:40.342860 17417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.340098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:40223 (local address 127.13.224.66:55387)
0806 13:06:40.340265 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:40.340276 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:40.340343 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:40.340502 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:40.340502 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:40.340504 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:40.340578 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.340580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.341219 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.341220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.342065 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.342068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.342614 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.342615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.342682 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.342686 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:06:40.342731 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 13:06:40.342771 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":13,"thread_start_us":96,"threads_started":1}
I20260806 13:06:40.342921 17399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.340232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:45267 (local address 127.13.224.67:45991)
0806 13:06:40.340291 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:40.340294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:40.340302 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:06:40.340332 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:40.340334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:40.340343 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:40.340397 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:40.340547 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.341080 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.342215 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.342584 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.342656 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.342664 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:40.342823 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 13:06:40.342862 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 13:06:40.342968 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.340096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:45991 (local address 127.13.224.66:45267)
0806 13:06:40.340156 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:40.340167 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:40.340212 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:40.340413 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:40.340413 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:40.340415 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:40.340489 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.340491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.341169 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.341170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.342130 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.342134 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.342671 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.342672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.342747 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.342751 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:06:40.342801 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 13:06:40.342840 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 13:06:40.343196 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "826c5f5bb7f54472b16700e53d0014b0" candidate_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37eb8641c2224cb490aed8424f8e9a22" is_pre_election: true
I20260806 13:06:40.343251 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "826c5f5bb7f54472b16700e53d0014b0" candidate_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5691b87b7f43439e83884208db071f32" is_pre_election: true
I20260806 13:06:40.343345 16884 raft_consensus.cc:2454] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9060a84263164dfcb4fe1f0b13cc4d7e in term 0.
I20260806 13:06:40.343379 17306 raft_consensus.cc:2454] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9060a84263164dfcb4fe1f0b13cc4d7e in term 0.
I20260806 13:06:40.343586 16982 leader_election.cc:302] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37eb8641c2224cb490aed8424f8e9a22, 9060a84263164dfcb4fe1f0b13cc4d7e; no voters:
I20260806 13:06:40.343776 17416 raft_consensus.cc:2781] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:06:40.343832 17416 raft_consensus.cc:482] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:06:40.343855 17416 raft_consensus.cc:3037] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:40.344935 17416 raft_consensus.cc:504] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } }
I20260806 13:06:40.345142 17416 leader_election.cc:288] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [CANDIDATE]: Term 1 election: Requested vote from peers 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991), 37eb8641c2224cb490aed8424f8e9a22 (127.13.224.65:40223)
I20260806 13:06:40.345225 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "826c5f5bb7f54472b16700e53d0014b0" candidate_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37eb8641c2224cb490aed8424f8e9a22"
I20260806 13:06:40.345288 16884 raft_consensus.cc:3037] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:40.345986 16884 raft_consensus.cc:2454] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9060a84263164dfcb4fe1f0b13cc4d7e in term 1.
I20260806 13:06:40.346119 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "826c5f5bb7f54472b16700e53d0014b0" candidate_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5691b87b7f43439e83884208db071f32"
I20260806 13:06:40.346153 16982 leader_election.cc:302] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37eb8641c2224cb490aed8424f8e9a22, 9060a84263164dfcb4fe1f0b13cc4d7e; no voters:
I20260806 13:06:40.346194 17306 raft_consensus.cc:3037] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:40.346226 17416 raft_consensus.cc:2781] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:40.346349 17416 raft_consensus.cc:686] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 LEADER]: Becoming Leader. State: Replica: 9060a84263164dfcb4fe1f0b13cc4d7e, State: Running, Role: LEADER
I20260806 13:06:40.346414 17416 consensus_queue.cc:237] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } }
I20260806 13:06:40.347059 17306 raft_consensus.cc:2454] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9060a84263164dfcb4fe1f0b13cc4d7e in term 1.
I20260806 13:06:40.347270 16651 catalog_manager.cc:5495] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9060a84263164dfcb4fe1f0b13cc4d7e (127.13.224.66). New cstate: current_term: 1 leader_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } health_report { overall_health: UNKNOWN } } }
Aug 06 13:06:40 dist-test-slave-q9h9 krb5kdc[16581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021599, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:40.380502 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.374600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:42265 (local address 127.0.0.1:58836)
0806 13:06:40.374744 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:40.374753 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:40.374824 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:40.374931 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:40.374932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:40.375225 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:40.375272 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.375274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.375871 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.375872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.376401 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.376402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.376687 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.376687 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.376730 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.377175 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:40.377179 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:40.378471 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:40.379692 (+ 1221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:40.379699 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:40.379701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:40.379881 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:40.380024 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:40.380024 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:40.380024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:40.380068 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:40.380242 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:40.380247 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:40.380284 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 13:06:40.380330 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 13:06:40.380415 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260806 13:06:40.380549 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.374669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58836 (local address 127.13.224.126:42265)
0806 13:06:40.374758 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:40.374759 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:40.374819 (+ 60us) server_negotiation.cc:407] Connection header received
0806 13:06:40.374874 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:40.374875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:40.374883 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:40.374920 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:40.375330 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.375801 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.376465 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.376637 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.376678 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.378550 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:40.378551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:40.378552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:40.378558 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:40.379628 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:40.379941 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:40.379942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:40.379942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:40.379988 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:40.380109 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:40.380109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:40.380109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:40.380197 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:40.380241 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:40.380356 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 13:06:40.380458 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":52}
I20260806 13:06:40.382372 16651 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:58836:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 13:06:40.382586 16651 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:06:40.383986 16855 tablet_service.cc:1467] Processing CreateTablet for tablet f2f5394030f6496ca6795170f84e8f6a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f8080759d1b24430a3ca0511c32a7402]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 13:06:40.383999 17274 tablet_service.cc:1467] Processing CreateTablet for tablet f2f5394030f6496ca6795170f84e8f6a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f8080759d1b24430a3ca0511c32a7402]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 13:06:40.384133 17274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f5394030f6496ca6795170f84e8f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:40.384151 16855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f5394030f6496ca6795170f84e8f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:40.385373 17407 tablet_bootstrap.cc:492] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22: Bootstrap starting.
I20260806 13:06:40.385468 17050 tablet_service.cc:1467] Processing CreateTablet for tablet f2f5394030f6496ca6795170f84e8f6a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f8080759d1b24430a3ca0511c32a7402]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 13:06:40.385524 17406 tablet_bootstrap.cc:492] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32: Bootstrap starting.
I20260806 13:06:40.385614 17050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2f5394030f6496ca6795170f84e8f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:40.385758 17407 tablet_bootstrap.cc:654] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:40.386053 17406 tablet_bootstrap.cc:654] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:40.386439 17407 tablet_bootstrap.cc:492] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22: No bootstrap required, opened a new log
I20260806 13:06:40.386495 17407 ts_tablet_manager.cc:1397] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:40.386649 17407 raft_consensus.cc:348] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.386720 17407 raft_consensus.cc:374] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:40.386739 17407 raft_consensus.cc:729] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37eb8641c2224cb490aed8424f8e9a22, State: Initialized, Role: FOLLOWER
I20260806 13:06:40.386790 17407 consensus_queue.cc:260] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.386900 17407 ts_tablet_manager.cc:1428] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:40.386936 17415 tablet_replica.cc:439] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } }
I20260806 13:06:40.387024 17415 tablet_replica.cc:442] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:06:40.387127 17404 tablet_bootstrap.cc:492] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e: Bootstrap starting.
I20260806 13:06:40.387310 17406 tablet_bootstrap.cc:492] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32: No bootstrap required, opened a new log
I20260806 13:06:40.387351 17406 ts_tablet_manager.cc:1397] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 13:06:40.387492 17406 raft_consensus.cc:348] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.387530 17404 tablet_bootstrap.cc:654] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:40.387547 17406 raft_consensus.cc:374] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:40.387562 17406 raft_consensus.cc:729] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5691b87b7f43439e83884208db071f32, State: Initialized, Role: FOLLOWER
I20260806 13:06:40.387595 17406 consensus_queue.cc:260] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.387670 17406 ts_tablet_manager.cc:1428] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:40.387777 17414 tablet_replica.cc:439] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } }
I20260806 13:06:40.387889 17414 tablet_replica.cc:442] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:06:40.388146 17404 tablet_bootstrap.cc:492] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e: No bootstrap required, opened a new log
I20260806 13:06:40.388193 17404 ts_tablet_manager.cc:1397] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:40.388339 17404 raft_consensus.cc:348] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.388401 17404 raft_consensus.cc:374] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:40.388419 17404 raft_consensus.cc:729] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9060a84263164dfcb4fe1f0b13cc4d7e, State: Initialized, Role: FOLLOWER
I20260806 13:06:40.388459 17404 consensus_queue.cc:260] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.388554 17404 ts_tablet_manager.cc:1428] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260806 13:06:40.388581 17375 tablet.cc:2369] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:06:40.388649 17416 tablet_replica.cc:439] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } }
I20260806 13:06:40.388705 17416 tablet_replica.cc:442] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:06:40.486522 17416 raft_consensus.cc:482] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:06:40.486618 17416 raft_consensus.cc:504] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.486773 17416 leader_election.cc:288] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37eb8641c2224cb490aed8424f8e9a22 (127.13.224.65:40223), 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991)
I20260806 13:06:40.487039 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f5394030f6496ca6795170f84e8f6a" candidate_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5691b87b7f43439e83884208db071f32" is_pre_election: true
I20260806 13:06:40.487039 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f5394030f6496ca6795170f84e8f6a" candidate_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37eb8641c2224cb490aed8424f8e9a22" is_pre_election: true
I20260806 13:06:40.487169 16884 raft_consensus.cc:2454] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9060a84263164dfcb4fe1f0b13cc4d7e in term 0.
I20260806 13:06:40.487169 17306 raft_consensus.cc:2454] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9060a84263164dfcb4fe1f0b13cc4d7e in term 0.
I20260806 13:06:40.487358 16982 leader_election.cc:302] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37eb8641c2224cb490aed8424f8e9a22, 9060a84263164dfcb4fe1f0b13cc4d7e; no voters:
I20260806 13:06:40.487485 17416 raft_consensus.cc:2781] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:06:40.487542 17416 raft_consensus.cc:482] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:06:40.487556 17416 raft_consensus.cc:3037] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:40.488267 17416 raft_consensus.cc:504] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.488387 17416 leader_election.cc:288] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [CANDIDATE]: Term 1 election: Requested vote from peers 37eb8641c2224cb490aed8424f8e9a22 (127.13.224.65:40223), 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991)
I20260806 13:06:40.488538 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f5394030f6496ca6795170f84e8f6a" candidate_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37eb8641c2224cb490aed8424f8e9a22"
I20260806 13:06:40.488586 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2f5394030f6496ca6795170f84e8f6a" candidate_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5691b87b7f43439e83884208db071f32"
I20260806 13:06:40.488634 16884 raft_consensus.cc:3037] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:40.488656 17306 raft_consensus.cc:3037] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:40.489248 17306 raft_consensus.cc:2454] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9060a84263164dfcb4fe1f0b13cc4d7e in term 1.
I20260806 13:06:40.489341 16884 raft_consensus.cc:2454] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9060a84263164dfcb4fe1f0b13cc4d7e in term 1.
I20260806 13:06:40.489447 16983 leader_election.cc:302] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5691b87b7f43439e83884208db071f32, 9060a84263164dfcb4fe1f0b13cc4d7e; no voters:
I20260806 13:06:40.489575 17416 raft_consensus.cc:2781] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:40.489667 17416 raft_consensus.cc:686] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 LEADER]: Becoming Leader. State: Replica: 9060a84263164dfcb4fe1f0b13cc4d7e, State: Running, Role: LEADER
I20260806 13:06:40.489748 17416 consensus_queue.cc:237] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.490073 17418 tablet_replica.cc:439] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e: TxnStatusTablet state changed. Reason: New leader 9060a84263164dfcb4fe1f0b13cc4d7e. Latest consensus state: current_term: 1 leader_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } }
I20260806 13:06:40.490163 17418 tablet_replica.cc:442] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e: This TxnStatusTablet replica's current role is: LEADER
I20260806 13:06:40.490360 17433 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 13:06:40.490463 17433 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 13:06:40.490427 16651 catalog_manager.cc:5495] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9060a84263164dfcb4fe1f0b13cc4d7e (127.13.224.66). New cstate: current_term: 1 leader_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } health_report { overall_health: UNKNOWN } } }
I20260806 13:06:40.503113 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.500396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:42751 (local address 127.0.0.1:41526)
0806 13:06:40.500479 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:40.500491 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:40.500544 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:40.500712 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:40.500713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:40.500714 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 13:06:40.500759 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.500762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.501629 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.501629 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.502231 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.502240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.502682 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.502683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.502723 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.502738 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 13:06:40.502994 (+ 256us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 13:06:40.502998 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:06:40.503021 (+ 23us) client_negotiation.cc:240] Negotiation successful
0806 13:06:40.503056 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 13:06:40.503155 17398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.500480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41526 (local address 127.13.224.66:42751)
0806 13:06:40.500557 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:40.500559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:40.500571 (+ 12us) server_negotiation.cc:407] Connection header received
0806 13:06:40.500606 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:40.500607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:40.500627 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:40.500708 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:06:40.500815 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.501553 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.502313 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.502609 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.502687 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.502819 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:06:40.502931 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:06:40.503014 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:40.503032 (+ 18us) server_negotiation.cc:299] Negotiation successful
0806 13:06:40.503064 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 13:06:40.504727 17306 raft_consensus.cc:1261] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Refusing update from remote peer 9060a84263164dfcb4fe1f0b13cc4d7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:40.504727 16884 raft_consensus.cc:1261] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Refusing update from remote peer 9060a84263164dfcb4fe1f0b13cc4d7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:40.505051 17416 consensus_queue.cc:1035] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:40.505174 17418 consensus_queue.cc:1035] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:40.506336 17414 tablet_replica.cc:439] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32: TxnStatusTablet state changed. Reason: New leader 9060a84263164dfcb4fe1f0b13cc4d7e. Latest consensus state: current_term: 1 leader_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } }
I20260806 13:06:40.506444 17414 tablet_replica.cc:442] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:06:40.506439 17415 tablet_replica.cc:439] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22: TxnStatusTablet state changed. Reason: New leader 9060a84263164dfcb4fe1f0b13cc4d7e. Latest consensus state: current_term: 1 leader_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } }
I20260806 13:06:40.506508 17415 tablet_replica.cc:442] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:06:40.507432 17416 tablet_replica.cc:439] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } }
I20260806 13:06:40.507468 17418 tablet_replica.cc:439] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } }
I20260806 13:06:40.507520 17416 tablet_replica.cc:442] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e: This TxnStatusTablet replica's current role is: LEADER
I20260806 13:06:40.507540 17418 tablet_replica.cc:442] T f2f5394030f6496ca6795170f84e8f6a P 9060a84263164dfcb4fe1f0b13cc4d7e: This TxnStatusTablet replica's current role is: LEADER
I20260806 13:06:40.507977 17415 tablet_replica.cc:439] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } }
I20260806 13:06:40.508049 17415 tablet_replica.cc:442] T f2f5394030f6496ca6795170f84e8f6a P 37eb8641c2224cb490aed8424f8e9a22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:06:40.508037 17414 tablet_replica.cc:439] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } } }
I20260806 13:06:40.508132 17414 tablet_replica.cc:442] T f2f5394030f6496ca6795170f84e8f6a P 5691b87b7f43439e83884208db071f32: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:06:40.509004 17441 mvcc.cc:205] Tried to move back new op lower bound from 7315544475665076224 to 7315544475606487040. Current Snapshot: MvccSnapshot[applied={T|T < 7315544475665076224}]
I20260806 13:06:40.512468 17398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.509967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41542 (local address 127.13.224.66:42751)
0806 13:06:40.510066 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:40.510068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:40.510077 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:06:40.510098 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:40.510099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:40.510109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:40.510174 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:06:40.510328 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.511013 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.511732 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.512036 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.512120 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.512248 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:06:40.512279 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:06:40.512339 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:40.512382 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 13:06:40.512401 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 13:06:40.517923 17398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.514896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:50743 (local address 127.13.224.66:42751)
0806 13:06:40.514935 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:40.514936 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:40.515030 (+ 94us) server_negotiation.cc:407] Connection header received
0806 13:06:40.515123 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:40.515124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:40.515130 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:40.515170 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:06:40.515794 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.516192 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.517264 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.517475 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.517524 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.517748 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:06:40.517769 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:06:40.517797 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:40.517870 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 13:06:40.517882 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:06:40.518067 17459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:40.514816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:42751 (local address 127.13.224.66:50743)
0806 13:06:40.514954 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:40.514965 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:40.515030 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:40.515178 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:40.515178 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:40.515180 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 13:06:40.515712 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.515715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.516264 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.516265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.517182 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:40.517185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:40.517546 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:40.517547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:40.517626 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:40.517645 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 13:06:40.517815 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 13:06:40.517817 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:06:40.517871 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 13:06:40.517910 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
I20260806 13:06:40.520033 16884 raft_consensus.cc:1261] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Refusing update from remote peer 9060a84263164dfcb4fe1f0b13cc4d7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:40.520025 17306 raft_consensus.cc:1261] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Refusing update from remote peer 9060a84263164dfcb4fe1f0b13cc4d7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:40.520218 17448 consensus_queue.cc:1035] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:40.520303 17418 consensus_queue.cc:1035] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:40.522004 17438 mvcc.cc:205] Tried to move back new op lower bound from 7315544475728887808 to 7315544475019370496. Current Snapshot: MvccSnapshot[applied={T|T < 7315544475728887808}]
I20260806 13:06:40.522428 17440 mvcc.cc:205] Tried to move back new op lower bound from 7315544475728887808 to 7315544475019370496. Current Snapshot: MvccSnapshot[applied={T|T < 7315544475728887808}]
I20260806 13:06:40.535387 16651 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:40.535467 16651 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:40.536170 16651 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6: Sending DeleteTablet for 3 replicas of tablet 826c5f5bb7f54472b16700e53d0014b0
I20260806 13:06:40.536442 17274 tablet_service.cc:1514] Processing DeleteTablet for tablet 826c5f5bb7f54472b16700e53d0014b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:40 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:42292
I20260806 13:06:40.536471 17051 tablet_service.cc:1514] Processing DeleteTablet for tablet 826c5f5bb7f54472b16700e53d0014b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:40 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:41522
I20260806 13:06:40.536700 17464 tablet_replica.cc:330] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e: stopping tablet replica
I20260806 13:06:40.536705 16855 tablet_service.cc:1514] Processing DeleteTablet for tablet 826c5f5bb7f54472b16700e53d0014b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:40 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:47422
I20260806 13:06:40.536823 17464 raft_consensus.cc:2229] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:06:40.536937 17464 raft_consensus.cc:2258] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:40.537132 16651 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:06:40.537273 16651 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:06:40.537381 17464 ts_tablet_manager.cc:1905] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:40.537215 17465 tablet_replica.cc:330] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22: stopping tablet replica
I20260806 13:06:40.537585 17465 raft_consensus.cc:2229] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:40.537691 17465 raft_consensus.cc:2258] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:40.538220 17463 tablet_replica.cc:330] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32: stopping tablet replica
I20260806 13:06:40.538370 17463 raft_consensus.cc:2229] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:40.538563 17463 raft_consensus.cc:2258] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:40.538662 17051 tablet_service.cc:1467] Processing CreateTablet for tablet 7d0f344dfeb14d91881ecfc80e0f0dc7 (DEFAULT_TABLE table=test-table [id=684e2fccbd054cf48b92d1c6cd69f715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:40.538784 17051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d0f344dfeb14d91881ecfc80e0f0dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:40.539106 17464 ts_tablet_manager.cc:1918] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:06:40.539179 17464 log.cc:1196] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/826c5f5bb7f54472b16700e53d0014b0
I20260806 13:06:40.539450 17464 ts_tablet_manager.cc:1939] T 826c5f5bb7f54472b16700e53d0014b0 P 9060a84263164dfcb4fe1f0b13cc4d7e: Deleting consensus metadata
I20260806 13:06:40.539851 16634 catalog_manager.cc:4841] TS 9060a84263164dfcb4fe1f0b13cc4d7e (127.13.224.66:42751): tablet 826c5f5bb7f54472b16700e53d0014b0 (table test-table [id=a2bd9e76171940fea1de6c493ba70a02]) successfully deleted
I20260806 13:06:40.539964 17404 tablet_bootstrap.cc:492] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e: Bootstrap starting.
I20260806 13:06:40.540274 16855 tablet_service.cc:1467] Processing CreateTablet for tablet 7d0f344dfeb14d91881ecfc80e0f0dc7 (DEFAULT_TABLE table=test-table [id=684e2fccbd054cf48b92d1c6cd69f715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:40.540360 17404 tablet_bootstrap.cc:654] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:40.540452 16855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d0f344dfeb14d91881ecfc80e0f0dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:40.541020 17404 tablet_bootstrap.cc:492] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e: No bootstrap required, opened a new log
I20260806 13:06:40.541072 17404 ts_tablet_manager.cc:1397] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:40.541200 17404 raft_consensus.cc:348] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.541272 17404 raft_consensus.cc:374] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:40.541290 17404 raft_consensus.cc:729] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9060a84263164dfcb4fe1f0b13cc4d7e, State: Initialized, Role: FOLLOWER
I20260806 13:06:40.541323 17404 consensus_queue.cc:260] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.541392 17404 ts_tablet_manager.cc:1428] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:40.541745 17407 tablet_bootstrap.cc:492] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22: Bootstrap starting.
I20260806 13:06:40.541913 17274 tablet_service.cc:1467] Processing CreateTablet for tablet 7d0f344dfeb14d91881ecfc80e0f0dc7 (DEFAULT_TABLE table=test-table [id=684e2fccbd054cf48b92d1c6cd69f715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:40.542052 17274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d0f344dfeb14d91881ecfc80e0f0dc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:40.542153 17407 tablet_bootstrap.cc:654] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:40.542812 17407 tablet_bootstrap.cc:492] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22: No bootstrap required, opened a new log
I20260806 13:06:40.542867 17407 ts_tablet_manager.cc:1397] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:40.542992 17407 raft_consensus.cc:348] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.543061 17407 raft_consensus.cc:374] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:40.543079 17407 raft_consensus.cc:729] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37eb8641c2224cb490aed8424f8e9a22, State: Initialized, Role: FOLLOWER
I20260806 13:06:40.543123 17407 consensus_queue.cc:260] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.543207 17407 ts_tablet_manager.cc:1428] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:40.543341 17406 tablet_bootstrap.cc:492] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32: Bootstrap starting.
I20260806 13:06:40.543752 17406 tablet_bootstrap.cc:654] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:40.543951 17465 ts_tablet_manager.cc:1905] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:40.544277 17463 ts_tablet_manager.cc:1905] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:40.544425 17406 tablet_bootstrap.cc:492] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32: No bootstrap required, opened a new log
I20260806 13:06:40.544471 17406 ts_tablet_manager.cc:1397] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:40.544607 17406 raft_consensus.cc:348] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.544670 17406 raft_consensus.cc:374] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:40.544692 17406 raft_consensus.cc:729] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5691b87b7f43439e83884208db071f32, State: Initialized, Role: FOLLOWER
I20260806 13:06:40.544740 17406 consensus_queue.cc:260] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.544834 17406 ts_tablet_manager.cc:1428] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:40.546582 17465 ts_tablet_manager.cc:1918] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:06:40.546659 17465 log.cc:1196] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/wal/wals/826c5f5bb7f54472b16700e53d0014b0
I20260806 13:06:40.546893 17465 ts_tablet_manager.cc:1939] T 826c5f5bb7f54472b16700e53d0014b0 P 37eb8641c2224cb490aed8424f8e9a22: Deleting consensus metadata
I20260806 13:06:40.547294 16629 catalog_manager.cc:4841] TS 37eb8641c2224cb490aed8424f8e9a22 (127.13.224.65:40223): tablet 826c5f5bb7f54472b16700e53d0014b0 (table test-table [id=a2bd9e76171940fea1de6c493ba70a02]) successfully deleted
I20260806 13:06:40.547369 17463 ts_tablet_manager.cc:1918] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:06:40.547429 17463 log.cc:1196] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/826c5f5bb7f54472b16700e53d0014b0
I20260806 13:06:40.547600 17463 ts_tablet_manager.cc:1939] T 826c5f5bb7f54472b16700e53d0014b0 P 5691b87b7f43439e83884208db071f32: Deleting consensus metadata
I20260806 13:06:40.547909 16632 catalog_manager.cc:4841] TS 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991): tablet 826c5f5bb7f54472b16700e53d0014b0 (table test-table [id=a2bd9e76171940fea1de6c493ba70a02]) successfully deleted
I20260806 13:06:40.643896 17448 raft_consensus.cc:482] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:06:40.643981 17448 raft_consensus.cc:504] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.644124 17448 leader_election.cc:288] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37eb8641c2224cb490aed8424f8e9a22 (127.13.224.65:40223), 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991)
I20260806 13:06:40.644347 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d0f344dfeb14d91881ecfc80e0f0dc7" candidate_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5691b87b7f43439e83884208db071f32" is_pre_election: true
I20260806 13:06:40.644359 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d0f344dfeb14d91881ecfc80e0f0dc7" candidate_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37eb8641c2224cb490aed8424f8e9a22" is_pre_election: true
I20260806 13:06:40.644450 17306 raft_consensus.cc:2454] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9060a84263164dfcb4fe1f0b13cc4d7e in term 0.
I20260806 13:06:40.644450 16884 raft_consensus.cc:2454] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9060a84263164dfcb4fe1f0b13cc4d7e in term 0.
I20260806 13:06:40.644608 16983 leader_election.cc:302] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5691b87b7f43439e83884208db071f32, 9060a84263164dfcb4fe1f0b13cc4d7e; no voters:
I20260806 13:06:40.644711 17448 raft_consensus.cc:2781] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:06:40.644753 17448 raft_consensus.cc:482] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:06:40.644770 17448 raft_consensus.cc:3037] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:40.645480 17448 raft_consensus.cc:504] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.645619 17448 leader_election.cc:288] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [CANDIDATE]: Term 1 election: Requested vote from peers 37eb8641c2224cb490aed8424f8e9a22 (127.13.224.65:40223), 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991)
I20260806 13:06:40.645757 16884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d0f344dfeb14d91881ecfc80e0f0dc7" candidate_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37eb8641c2224cb490aed8424f8e9a22"
I20260806 13:06:40.645830 16884 raft_consensus.cc:3037] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:40.645812 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d0f344dfeb14d91881ecfc80e0f0dc7" candidate_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5691b87b7f43439e83884208db071f32"
I20260806 13:06:40.645913 17306 raft_consensus.cc:3037] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:40.646436 16884 raft_consensus.cc:2454] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9060a84263164dfcb4fe1f0b13cc4d7e in term 1.
I20260806 13:06:40.646624 16982 leader_election.cc:302] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37eb8641c2224cb490aed8424f8e9a22, 9060a84263164dfcb4fe1f0b13cc4d7e; no voters:
I20260806 13:06:40.646705 17448 raft_consensus.cc:2781] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:40.646754 17448 raft_consensus.cc:686] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 LEADER]: Becoming Leader. State: Replica: 9060a84263164dfcb4fe1f0b13cc4d7e, State: Running, Role: LEADER
I20260806 13:06:40.646749 17306 raft_consensus.cc:2454] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9060a84263164dfcb4fe1f0b13cc4d7e in term 1.
I20260806 13:06:40.646800 17448 consensus_queue.cc:237] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:40.647320 16651 catalog_manager.cc:5495] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9060a84263164dfcb4fe1f0b13cc4d7e (127.13.224.66). New cstate: current_term: 1 leader_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } health_report { overall_health: UNKNOWN } } }
I20260806 13:06:40.664108 16884 raft_consensus.cc:1261] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Refusing update from remote peer 9060a84263164dfcb4fe1f0b13cc4d7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:40.664095 17306 raft_consensus.cc:1261] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Refusing update from remote peer 9060a84263164dfcb4fe1f0b13cc4d7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:40.664302 17448 consensus_queue.cc:1035] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:40.664448 17471 consensus_queue.cc:1035] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 13:06:40.668762 17438 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 13:06:42.190160 16650 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:42.190284 16650 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:42.191084 16650 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6: Sending DeleteTablet for 3 replicas of tablet 7d0f344dfeb14d91881ecfc80e0f0dc7
I20260806 13:06:42.191368 17051 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d0f344dfeb14d91881ecfc80e0f0dc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:42 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:41522
I20260806 13:06:42.191411 17274 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d0f344dfeb14d91881ecfc80e0f0dc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:42 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:42292
I20260806 13:06:42.191421 16855 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d0f344dfeb14d91881ecfc80e0f0dc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:42 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:47422
I20260806 13:06:42.191859 17516 tablet_replica.cc:330] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32: stopping tablet replica
I20260806 13:06:42.191912 17516 raft_consensus.cc:2229] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:42.191941 17516 raft_consensus.cc:2258] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:42.192023 17515 tablet_replica.cc:330] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22: stopping tablet replica
I20260806 13:06:42.192063 17515 raft_consensus.cc:2229] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:42.192087 17515 raft_consensus.cc:2258] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:42.192229 17516 ts_tablet_manager.cc:1905] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:42.192284 17515 ts_tablet_manager.cc:1905] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:42.193970 17516 ts_tablet_manager.cc:1918] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 13:06:42.194051 17516 log.cc:1196] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/7d0f344dfeb14d91881ecfc80e0f0dc7
I20260806 13:06:42.194250 17516 ts_tablet_manager.cc:1939] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 5691b87b7f43439e83884208db071f32: Deleting consensus metadata
I20260806 13:06:42.194278 17514 tablet_replica.cc:330] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e: stopping tablet replica
I20260806 13:06:42.194334 17514 raft_consensus.cc:2229] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:06:42.194393 17514 raft_consensus.cc:2258] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:42.194648 17514 ts_tablet_manager.cc:1905] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:42.194664 16632 catalog_manager.cc:4841] TS 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991): tablet 7d0f344dfeb14d91881ecfc80e0f0dc7 (table test-table [id=684e2fccbd054cf48b92d1c6cd69f715]) successfully deleted
I20260806 13:06:42.194698 17515 ts_tablet_manager.cc:1918] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 13:06:42.194759 17515 log.cc:1196] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/wal/wals/7d0f344dfeb14d91881ecfc80e0f0dc7
I20260806 13:06:42.194978 17515 ts_tablet_manager.cc:1939] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 37eb8641c2224cb490aed8424f8e9a22: Deleting consensus metadata
I20260806 13:06:42.195271 16629 catalog_manager.cc:4841] TS 37eb8641c2224cb490aed8424f8e9a22 (127.13.224.65:40223): tablet 7d0f344dfeb14d91881ecfc80e0f0dc7 (table test-table [id=684e2fccbd054cf48b92d1c6cd69f715]) successfully deleted
I20260806 13:06:42.195989 17514 ts_tablet_manager.cc:1918] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 13:06:42.196041 17514 log.cc:1196] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/7d0f344dfeb14d91881ecfc80e0f0dc7
I20260806 13:06:42.196246 17514 ts_tablet_manager.cc:1939] T 7d0f344dfeb14d91881ecfc80e0f0dc7 P 9060a84263164dfcb4fe1f0b13cc4d7e: Deleting consensus metadata
I20260806 13:06:42.196563 16634 catalog_manager.cc:4841] TS 9060a84263164dfcb4fe1f0b13cc4d7e (127.13.224.66:42751): tablet 7d0f344dfeb14d91881ecfc80e0f0dc7 (table test-table [id=684e2fccbd054cf48b92d1c6cd69f715]) successfully deleted
Aug 06 13:06:42 dist-test-slave-q9h9 krb5kdc[16581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021602, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 13:06:42 dist-test-slave-q9h9 krb5kdc[16581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021602, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:42.216457 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:42.210066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.13.224.126:42265)
0806 13:06:42.210186 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:42.210188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:42.210218 (+ 30us) server_negotiation.cc:407] Connection header received
0806 13:06:42.210292 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:42.210293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:42.210302 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:42.210348 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:42.210838 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.211321 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.212060 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.212258 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.212306 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:42.214401 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:42.214402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:42.214403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:42.214409 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:42.215617 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:42.215904 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:42.215905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:42.215905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:42.215954 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:42.216088 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:42.216088 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:42.216088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:42.216178 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:42.216246 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:42.216326 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 13:06:42.216385 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":74,"threads_started":1}
I20260806 13:06:42.218055 16650 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:06:42.218199 16650 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:06:42.219429 17051 tablet_service.cc:1467] Processing CreateTablet for tablet e8205a7e9c3c470f8c7c9f39f0294c1c (DEFAULT_TABLE table=test-table [id=861b606a72dd4b618876b6652ff6273c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:42.219429 16855 tablet_service.cc:1467] Processing CreateTablet for tablet e8205a7e9c3c470f8c7c9f39f0294c1c (DEFAULT_TABLE table=test-table [id=861b606a72dd4b618876b6652ff6273c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:42.219553 17051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8205a7e9c3c470f8c7c9f39f0294c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:42.219553 16855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8205a7e9c3c470f8c7c9f39f0294c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:42.220839 17535 tablet_bootstrap.cc:492] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22: Bootstrap starting.
I20260806 13:06:42.220872 17274 tablet_service.cc:1467] Processing CreateTablet for tablet e8205a7e9c3c470f8c7c9f39f0294c1c (DEFAULT_TABLE table=test-table [id=861b606a72dd4b618876b6652ff6273c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:42.220993 17274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8205a7e9c3c470f8c7c9f39f0294c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:42.221201 17535 tablet_bootstrap.cc:654] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:42.221470 17532 tablet_bootstrap.cc:492] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e: Bootstrap starting.
I20260806 13:06:42.221782 17532 tablet_bootstrap.cc:654] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:42.221793 17535 tablet_bootstrap.cc:492] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22: No bootstrap required, opened a new log
I20260806 13:06:42.221904 17535 ts_tablet_manager.cc:1397] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 13:06:42.222085 17539 tablet_bootstrap.cc:492] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32: Bootstrap starting.
I20260806 13:06:42.222117 17535 raft_consensus.cc:348] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.222252 17535 raft_consensus.cc:374] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:42.222278 17535 raft_consensus.cc:729] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37eb8641c2224cb490aed8424f8e9a22, State: Initialized, Role: FOLLOWER
I20260806 13:06:42.222337 17535 consensus_queue.cc:260] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.222482 17539 tablet_bootstrap.cc:654] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:42.222564 17532 tablet_bootstrap.cc:492] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e: No bootstrap required, opened a new log
I20260806 13:06:42.222609 17532 ts_tablet_manager.cc:1397] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:42.222723 17532 raft_consensus.cc:348] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.222805 17532 raft_consensus.cc:374] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:42.222831 17532 raft_consensus.cc:729] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9060a84263164dfcb4fe1f0b13cc4d7e, State: Initialized, Role: FOLLOWER
I20260806 13:06:42.222863 17535 ts_tablet_manager.cc:1428] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 13:06:42.222892 17532 consensus_queue.cc:260] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.222980 17539 tablet_bootstrap.cc:492] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32: No bootstrap required, opened a new log
I20260806 13:06:42.223022 17539 ts_tablet_manager.cc:1397] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:42.223095 17532 ts_tablet_manager.cc:1428] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:42.223142 17539 raft_consensus.cc:348] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.223220 17539 raft_consensus.cc:374] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:42.223238 17539 raft_consensus.cc:729] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5691b87b7f43439e83884208db071f32, State: Initialized, Role: FOLLOWER
I20260806 13:06:42.223284 17539 consensus_queue.cc:260] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.223448 17539 ts_tablet_manager.cc:1428] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:42.414295 17542 raft_consensus.cc:482] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:06:42.414391 17542 raft_consensus.cc:504] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.414752 17542 leader_election.cc:288] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9060a84263164dfcb4fe1f0b13cc4d7e (127.13.224.66:42751), 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991)
I20260806 13:06:42.417761 17546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:42.414949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:45991 (local address 127.13.224.65:40889)
0806 13:06:42.415124 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:42.415134 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:42.415190 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:42.415361 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:42.415361 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:42.415367 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:42.415438 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:42.415440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.416082 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.416083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:42.416870 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:42.416872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.417511 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.417512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:42.417575 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:42.417578 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:06:42.417623 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 13:06:42.417672 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":2,"thread_start_us":106,"threads_started":1}
I20260806 13:06:42.417960 17549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:42.415050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:40889 (local address 127.13.224.67:45991)
0806 13:06:42.415231 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:42.415233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:42.415240 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:06:42.415267 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:42.415268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:42.415278 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:42.415340 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:42.415514 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.416000 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.416961 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.417428 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.417493 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:42.417502 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:42.417640 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 13:06:42.417677 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":117,"threads_started":1}
I20260806 13:06:42.418107 17547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:42.414920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:42751 (local address 127.13.224.65:46689)
0806 13:06:42.415114 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:42.415126 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:42.415189 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:42.415360 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:42.415361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:42.415363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:42.415448 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:42.415450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.416094 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.416095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:42.417301 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:42.417304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.417848 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.417849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:42.417926 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:42.417931 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:06:42.417968 (+ 37us) client_negotiation.cc:240] Negotiation successful
0806 13:06:42.418006 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":5,"thread_start_us":115,"threads_started":1}
I20260806 13:06:42.418172 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8205a7e9c3c470f8c7c9f39f0294c1c" candidate_uuid: "37eb8641c2224cb490aed8424f8e9a22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5691b87b7f43439e83884208db071f32" is_pre_election: true
I20260806 13:06:42.418244 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:42.415019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:46689 (local address 127.13.224.66:42751)
0806 13:06:42.415158 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:42.415159 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:42.415172 (+ 13us) server_negotiation.cc:407] Connection header received
0806 13:06:42.415268 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:42.415269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:42.415279 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:42.415338 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:42.415527 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.416007 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.417390 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.417760 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.418112 (+ 352us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:42.418124 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:42.418147 (+ 23us) server_negotiation.cc:299] Negotiation successful
0806 13:06:42.418189 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260806 13:06:42.418277 17306 raft_consensus.cc:2454] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37eb8641c2224cb490aed8424f8e9a22 in term 0.
I20260806 13:06:42.418439 17095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8205a7e9c3c470f8c7c9f39f0294c1c" candidate_uuid: "37eb8641c2224cb490aed8424f8e9a22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" is_pre_election: true
I20260806 13:06:42.418478 16787 leader_election.cc:302] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37eb8641c2224cb490aed8424f8e9a22, 5691b87b7f43439e83884208db071f32; no voters:
I20260806 13:06:42.418570 17095 raft_consensus.cc:2454] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37eb8641c2224cb490aed8424f8e9a22 in term 0.
I20260806 13:06:42.418619 17542 raft_consensus.cc:2781] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:06:42.418659 17542 raft_consensus.cc:482] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:06:42.418679 17542 raft_consensus.cc:3037] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:42.419363 17542 raft_consensus.cc:504] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.419484 17542 leader_election.cc:288] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [CANDIDATE]: Term 1 election: Requested vote from peers 9060a84263164dfcb4fe1f0b13cc4d7e (127.13.224.66:42751), 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991)
I20260806 13:06:42.419631 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8205a7e9c3c470f8c7c9f39f0294c1c" candidate_uuid: "37eb8641c2224cb490aed8424f8e9a22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5691b87b7f43439e83884208db071f32"
I20260806 13:06:42.419652 17095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8205a7e9c3c470f8c7c9f39f0294c1c" candidate_uuid: "37eb8641c2224cb490aed8424f8e9a22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e"
I20260806 13:06:42.419708 17306 raft_consensus.cc:3037] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:42.419714 17095 raft_consensus.cc:3037] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:42.420261 17306 raft_consensus.cc:2454] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37eb8641c2224cb490aed8424f8e9a22 in term 1.
I20260806 13:06:42.420374 17095 raft_consensus.cc:2454] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37eb8641c2224cb490aed8424f8e9a22 in term 1.
I20260806 13:06:42.420428 16787 leader_election.cc:302] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37eb8641c2224cb490aed8424f8e9a22, 5691b87b7f43439e83884208db071f32; no voters:
I20260806 13:06:42.420516 17542 raft_consensus.cc:2781] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:42.420653 17542 raft_consensus.cc:686] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [term 1 LEADER]: Becoming Leader. State: Replica: 37eb8641c2224cb490aed8424f8e9a22, State: Running, Role: LEADER
I20260806 13:06:42.420745 17542 consensus_queue.cc:237] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.421417 16651 catalog_manager.cc:5495] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 37eb8641c2224cb490aed8424f8e9a22 (127.13.224.65). New cstate: current_term: 1 leader_uuid: "37eb8641c2224cb490aed8424f8e9a22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } health_report { overall_health: UNKNOWN } } }
I20260806 13:06:42.450533 17553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:42.448088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47438 (local address 127.13.224.65:40223)
0806 13:06:42.448275 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:42.448277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:42.448284 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:06:42.448325 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:42.448325 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:42.448347 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:42.448422 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:06:42.448551 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.449051 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.449787 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.449983 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.450066 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:42.450234 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:06:42.450341 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:06:42.450397 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:42.450425 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 13:06:42.450442 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":114,"threads_started":1}
I20260806 13:06:42.455436 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:42.453155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:51259 (local address 127.13.224.66:42751)
0806 13:06:42.453222 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:42.453224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:42.453279 (+ 55us) server_negotiation.cc:407] Connection header received
0806 13:06:42.453340 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:42.453341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:42.453352 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:42.453399 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:06:42.453520 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.453998 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.454770 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.454976 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.455029 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:42.455206 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:06:42.455236 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:06:42.455278 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:42.455336 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 13:06:42.455359 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 13:06:42.455461 17557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:42.453053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:42751 (local address 127.13.224.65:51259)
0806 13:06:42.453209 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:42.453220 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:42.453294 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:42.453399 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:42.453399 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:42.453401 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 13:06:42.453460 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:42.453461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.454095 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.454096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:42.454680 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:42.454682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.455047 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.455048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:42.455113 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:42.455128 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 13:06:42.455292 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 13:06:42.455294 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:06:42.455335 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 13:06:42.455380 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260806 13:06:42.457441 17306 raft_consensus.cc:1261] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Refusing update from remote peer 37eb8641c2224cb490aed8424f8e9a22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:42.457573 17095 raft_consensus.cc:1261] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Refusing update from remote peer 37eb8641c2224cb490aed8424f8e9a22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:42.457742 17550 consensus_queue.cc:1035] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:42.457875 17542 consensus_queue.cc:1035] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:42.464440 17553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:42.462346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:58579 (local address 127.13.224.65:40223)
0806 13:06:42.462406 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:42.462407 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:42.462531 (+ 124us) server_negotiation.cc:407] Connection header received
0806 13:06:42.462559 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:42.462559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:42.462570 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:42.462620 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:06:42.462695 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.463121 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.463781 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.464000 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.464076 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:42.464202 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:06:42.464235 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:06:42.464281 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:42.464338 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 13:06:42.464358 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 13:06:42.464466 17567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:42.462289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:40223 (local address 127.13.224.66:58579)
0806 13:06:42.462457 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:42.462466 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:42.462520 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:42.462624 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:42.462625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:42.462626 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 13:06:42.462662 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:42.462663 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.463172 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.463172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:42.463710 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:42.463711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:42.464080 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:42.464081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:42.464142 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:42.464146 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 13:06:42.464293 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 13:06:42.464295 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:06:42.464334 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 13:06:42.464379 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":74,"threads_started":1}
I20260806 13:06:42.470880 16650 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:42.470968 16650 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:42.471573 16650 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6: Sending DeleteTablet for 3 replicas of tablet e8205a7e9c3c470f8c7c9f39f0294c1c
I20260806 13:06:42.471808 16855 tablet_service.cc:1514] Processing DeleteTablet for tablet e8205a7e9c3c470f8c7c9f39f0294c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:42 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:47422
I20260806 13:06:42.471845 17051 tablet_service.cc:1514] Processing DeleteTablet for tablet e8205a7e9c3c470f8c7c9f39f0294c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:42 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:41522
I20260806 13:06:42.471848 17274 tablet_service.cc:1514] Processing DeleteTablet for tablet e8205a7e9c3c470f8c7c9f39f0294c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:42 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:42292
I20260806 13:06:42.471896 17515 tablet_replica.cc:330] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22: stopping tablet replica
I20260806 13:06:42.471928 17516 tablet_replica.cc:330] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32: stopping tablet replica
I20260806 13:06:42.471930 17514 tablet_replica.cc:330] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e: stopping tablet replica
I20260806 13:06:42.471940 17515 raft_consensus.cc:2229] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:06:42.471971 17516 raft_consensus.cc:2229] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:42.471972 17514 raft_consensus.cc:2229] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:42.472003 17514 raft_consensus.cc:2258] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:42.472007 17516 raft_consensus.cc:2258] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:42.472014 17515 raft_consensus.cc:2258] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:42.472321 17516 ts_tablet_manager.cc:1905] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:42.472515 17515 ts_tablet_manager.cc:1905] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:42.472460 16650 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:06:42.472595 16650 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:06:42.473731 17274 tablet_service.cc:1467] Processing CreateTablet for tablet ef82a818c77c4e6eb088b580eaceffb6 (DEFAULT_TABLE table=test-table [id=fd12440797104a40a7d589e6148f026d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:42.474032 17051 tablet_service.cc:1467] Processing CreateTablet for tablet ef82a818c77c4e6eb088b580eaceffb6 (DEFAULT_TABLE table=test-table [id=fd12440797104a40a7d589e6148f026d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:42.474043 17515 ts_tablet_manager.cc:1918] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:06:42.474109 17515 log.cc:1196] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-0/wal/wals/e8205a7e9c3c470f8c7c9f39f0294c1c
I20260806 13:06:42.474155 17051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef82a818c77c4e6eb088b580eaceffb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:42.474144 17516 ts_tablet_manager.cc:1918] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:06:42.474189 17516 log.cc:1196] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/e8205a7e9c3c470f8c7c9f39f0294c1c
I20260806 13:06:42.474340 17515 ts_tablet_manager.cc:1939] T e8205a7e9c3c470f8c7c9f39f0294c1c P 37eb8641c2224cb490aed8424f8e9a22: Deleting consensus metadata
I20260806 13:06:42.474395 17516 ts_tablet_manager.cc:1939] T e8205a7e9c3c470f8c7c9f39f0294c1c P 5691b87b7f43439e83884208db071f32: Deleting consensus metadata
I20260806 13:06:42.474054 16855 tablet_service.cc:1467] Processing CreateTablet for tablet ef82a818c77c4e6eb088b580eaceffb6 (DEFAULT_TABLE table=test-table [id=fd12440797104a40a7d589e6148f026d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:42.474677 16855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef82a818c77c4e6eb088b580eaceffb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:42.475590 17532 tablet_bootstrap.cc:492] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e: Bootstrap starting.
I20260806 13:06:42.475835 17535 tablet_bootstrap.cc:492] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22: Bootstrap starting.
I20260806 13:06:42.475907 17532 tablet_bootstrap.cc:654] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:42.476214 17535 tablet_bootstrap.cc:654] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:42.476198 17514 ts_tablet_manager.cc:1905] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:42.476598 16629 catalog_manager.cc:4841] TS 37eb8641c2224cb490aed8424f8e9a22 (127.13.224.65:40223): tablet e8205a7e9c3c470f8c7c9f39f0294c1c (table test-table [id=861b606a72dd4b618876b6652ff6273c]) successfully deleted
I20260806 13:06:42.476745 17274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef82a818c77c4e6eb088b580eaceffb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:42.477576 17514 ts_tablet_manager.cc:1918] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:06:42.477635 17514 log.cc:1196] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/e8205a7e9c3c470f8c7c9f39f0294c1c
I20260806 13:06:42.477846 17514 ts_tablet_manager.cc:1939] T e8205a7e9c3c470f8c7c9f39f0294c1c P 9060a84263164dfcb4fe1f0b13cc4d7e: Deleting consensus metadata
I20260806 13:06:42.477902 17539 tablet_bootstrap.cc:492] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32: Bootstrap starting.
I20260806 13:06:42.478143 17535 tablet_bootstrap.cc:492] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22: No bootstrap required, opened a new log
I20260806 13:06:42.478191 17535 ts_tablet_manager.cc:1397] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 13:06:42.478308 17532 tablet_bootstrap.cc:492] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e: No bootstrap required, opened a new log
I20260806 13:06:42.478344 17532 ts_tablet_manager.cc:1397] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 13:06:42.478361 17535 raft_consensus.cc:348] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.478425 17535 raft_consensus.cc:374] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:42.478442 17535 raft_consensus.cc:729] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37eb8641c2224cb490aed8424f8e9a22, State: Initialized, Role: FOLLOWER
I20260806 13:06:42.478431 17532 raft_consensus.cc:348] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.478483 17532 raft_consensus.cc:374] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:42.478484 17539 tablet_bootstrap.cc:654] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:42.478502 17532 raft_consensus.cc:729] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9060a84263164dfcb4fe1f0b13cc4d7e, State: Initialized, Role: FOLLOWER
I20260806 13:06:42.478482 17535 consensus_queue.cc:260] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.478538 17532 consensus_queue.cc:260] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.478581 17535 ts_tablet_manager.cc:1428] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:42.478621 17532 ts_tablet_manager.cc:1428] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:42.478873 16634 catalog_manager.cc:4841] TS 9060a84263164dfcb4fe1f0b13cc4d7e (127.13.224.66:42751): tablet e8205a7e9c3c470f8c7c9f39f0294c1c (table test-table [id=861b606a72dd4b618876b6652ff6273c]) successfully deleted
I20260806 13:06:42.478989 16632 catalog_manager.cc:4841] TS 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991): tablet e8205a7e9c3c470f8c7c9f39f0294c1c (table test-table [id=861b606a72dd4b618876b6652ff6273c]) successfully deleted
I20260806 13:06:42.479461 17539 tablet_bootstrap.cc:492] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32: No bootstrap required, opened a new log
I20260806 13:06:42.479509 17539 ts_tablet_manager.cc:1397] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 13:06:42.479641 17539 raft_consensus.cc:348] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.479717 17539 raft_consensus.cc:374] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:42.479753 17539 raft_consensus.cc:729] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5691b87b7f43439e83884208db071f32, State: Initialized, Role: FOLLOWER
I20260806 13:06:42.479791 17539 consensus_queue.cc:260] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.479879 17539 ts_tablet_manager.cc:1428] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:42.711290 17565 raft_consensus.cc:482] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:06:42.711388 17565 raft_consensus.cc:504] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.711575 17565 leader_election.cc:288] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9060a84263164dfcb4fe1f0b13cc4d7e (127.13.224.66:42751), 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991)
I20260806 13:06:42.711784 17095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef82a818c77c4e6eb088b580eaceffb6" candidate_uuid: "37eb8641c2224cb490aed8424f8e9a22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" is_pre_election: true
I20260806 13:06:42.711786 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef82a818c77c4e6eb088b580eaceffb6" candidate_uuid: "37eb8641c2224cb490aed8424f8e9a22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5691b87b7f43439e83884208db071f32" is_pre_election: true
I20260806 13:06:42.711880 17095 raft_consensus.cc:2454] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37eb8641c2224cb490aed8424f8e9a22 in term 0.
I20260806 13:06:42.711889 17306 raft_consensus.cc:2454] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37eb8641c2224cb490aed8424f8e9a22 in term 0.
I20260806 13:06:42.712030 16787 leader_election.cc:302] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37eb8641c2224cb490aed8424f8e9a22, 5691b87b7f43439e83884208db071f32; no voters:
I20260806 13:06:42.712141 17565 raft_consensus.cc:2781] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:06:42.712184 17565 raft_consensus.cc:482] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:06:42.712210 17565 raft_consensus.cc:3037] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:42.712991 17565 raft_consensus.cc:504] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.713147 17565 leader_election.cc:288] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [CANDIDATE]: Term 1 election: Requested vote from peers 9060a84263164dfcb4fe1f0b13cc4d7e (127.13.224.66:42751), 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991)
I20260806 13:06:42.713295 17095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef82a818c77c4e6eb088b580eaceffb6" candidate_uuid: "37eb8641c2224cb490aed8424f8e9a22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e"
I20260806 13:06:42.713379 17095 raft_consensus.cc:3037] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:42.713362 17306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef82a818c77c4e6eb088b580eaceffb6" candidate_uuid: "37eb8641c2224cb490aed8424f8e9a22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5691b87b7f43439e83884208db071f32"
I20260806 13:06:42.713457 17306 raft_consensus.cc:3037] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:42.713992 17095 raft_consensus.cc:2454] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37eb8641c2224cb490aed8424f8e9a22 in term 1.
I20260806 13:06:42.714109 17306 raft_consensus.cc:2454] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37eb8641c2224cb490aed8424f8e9a22 in term 1.
I20260806 13:06:42.714181 16789 leader_election.cc:302] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37eb8641c2224cb490aed8424f8e9a22, 9060a84263164dfcb4fe1f0b13cc4d7e; no voters:
I20260806 13:06:42.714269 17565 raft_consensus.cc:2781] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:42.714318 17565 raft_consensus.cc:686] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [term 1 LEADER]: Becoming Leader. State: Replica: 37eb8641c2224cb490aed8424f8e9a22, State: Running, Role: LEADER
I20260806 13:06:42.714362 17565 consensus_queue.cc:237] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } }
I20260806 13:06:42.714972 16650 catalog_manager.cc:5495] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 37eb8641c2224cb490aed8424f8e9a22 (127.13.224.65). New cstate: current_term: 1 leader_uuid: "37eb8641c2224cb490aed8424f8e9a22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37eb8641c2224cb490aed8424f8e9a22" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 } health_report { overall_health: UNKNOWN } } }
I20260806 13:06:42.762835 17306 raft_consensus.cc:1261] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Refusing update from remote peer 37eb8641c2224cb490aed8424f8e9a22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:42.762898 17095 raft_consensus.cc:1261] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Refusing update from remote peer 37eb8641c2224cb490aed8424f8e9a22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:42.763078 17550 consensus_queue.cc:1035] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9060a84263164dfcb4fe1f0b13cc4d7e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:42.763218 17565 consensus_queue.cc:1035] T ef82a818c77c4e6eb088b580eaceffb6 P 37eb8641c2224cb490aed8424f8e9a22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5691b87b7f43439e83884208db071f32" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 13:06:42.767190 17484 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 13:06:44.287321 16651 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:44.287417 16651 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:44.288218 16651 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fb4104091384da49e66ac27dd40c2f6: Sending DeleteTablet for 3 replicas of tablet ef82a818c77c4e6eb088b580eaceffb6
I20260806 13:06:44.288509 16855 tablet_service.cc:1514] Processing DeleteTablet for tablet ef82a818c77c4e6eb088b580eaceffb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:44 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:47422
I20260806 13:06:44.288524 17051 tablet_service.cc:1514] Processing DeleteTablet for tablet ef82a818c77c4e6eb088b580eaceffb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:44 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:41522
I20260806 13:06:44.288656 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 16761
I20260806 13:06:44.288691 17274 tablet_service.cc:1514] Processing DeleteTablet for tablet ef82a818c77c4e6eb088b580eaceffb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:44 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:42292
I20260806 13:06:44.293885 17608 tablet_replica.cc:330] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32: stopping tablet replica
I20260806 13:06:44.293948 17608 raft_consensus.cc:2229] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:44.293975 17608 raft_consensus.cc:2258] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:44.294106 17607 tablet_replica.cc:330] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e: stopping tablet replica
I20260806 13:06:44.294332 17608 ts_tablet_manager.cc:1905] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:44.295644 17607 raft_consensus.cc:2229] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:44.295706 17607 raft_consensus.cc:2258] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:44.295845 17608 ts_tablet_manager.cc:1918] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 13:06:44.295897 17608 log.cc:1196] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/ef82a818c77c4e6eb088b580eaceffb6
I20260806 13:06:44.296082 17608 ts_tablet_manager.cc:1939] T ef82a818c77c4e6eb088b580eaceffb6 P 5691b87b7f43439e83884208db071f32: Deleting consensus metadata
I20260806 13:06:44.296397 16632 catalog_manager.cc:4841] TS 5691b87b7f43439e83884208db071f32 (127.13.224.67:45991): tablet ef82a818c77c4e6eb088b580eaceffb6 (table test-table [id=fd12440797104a40a7d589e6148f026d]) successfully deleted
I20260806 13:06:44.296543 17607 ts_tablet_manager.cc:1905] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:44.297643 17607 ts_tablet_manager.cc:1918] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 13:06:44.297696 17607 log.cc:1196] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/ef82a818c77c4e6eb088b580eaceffb6
I20260806 13:06:44.297889 17607 ts_tablet_manager.cc:1939] T ef82a818c77c4e6eb088b580eaceffb6 P 9060a84263164dfcb4fe1f0b13cc4d7e: Deleting consensus metadata
I20260806 13:06:44.298350 16634 catalog_manager.cc:4841] TS 9060a84263164dfcb4fe1f0b13cc4d7e (127.13.224.66:42751): tablet ef82a818c77c4e6eb088b580eaceffb6 (table test-table [id=fd12440797104a40a7d589e6148f026d]) successfully deleted
W20260806 13:06:44.303440 16629 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:40223 (error 108)
I20260806 13:06:44.304046 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 16959
I20260806 13:06:44.304087 17613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:44.303871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:40223 (local address 127.0.0.1:47440)
0806 13:06:44.303995 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:44.304047 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:40223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":71,"threads_started":1}
W20260806 13:06:44.311887 16629 catalog_manager.cc:4584] TS 37eb8641c2224cb490aed8424f8e9a22 (127.13.224.65:40223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ef82a818c77c4e6eb088b580eaceffb6: Network error: Client connection negotiation failed: client connection to 127.13.224.65:40223: connect: Connection refused (error 111)
I20260806 13:06:44.321888 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 17173
I20260806 13:06:44.330844 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 16603
2026-08-06T13:06:44Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6003 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:06:44 dist-test-slave-q9h9 krb5kdc[17619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:06:44 dist-test-slave-q9h9 krb5kdc[17619](info): set up 2 sockets
Aug 06 13:06:44 dist-test-slave-q9h9 krb5kdc[17619](info): commencing operation
krb5kdc: starting...
W20260806 13:06:45.459648 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:06:45 dist-test-slave-q9h9 krb5kdc[17619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:06:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:06:45.566838 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:33847
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:39043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:33847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:45.646760 17635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:45.646947 17635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:45.646975 17635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:45.648068 17635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:45.648106 17635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:45.648125 17635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:45.648140 17635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:45.648156 17635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:06:45.649598 17635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:39043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:33847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:33847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:45.649837 17635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:45.650064 17635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 13:06:45.654665 17635 server_base.cc:1034] running on GCE node
W20260806 13:06:45.654619 17645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:45.654640 17643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:45.654615 17641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:45.655030 17635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:45.655298 17635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:45.656448 17635 hybrid_clock.cc:648] HybridClock initialized: now 1786021605656436 us; error 30 us; skew 500 ppm
Aug 06 13:06:45 dist-test-slave-q9h9 krb5kdc[17619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021605, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:45.658733 17635 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:06:45.659116 17635 webserver.cc:460] Webserver started at http://127.13.224.126:39511/ using document root <none> and password file <none>
I20260806 13:06:45.659358 17635 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:45.659407 17635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:45.659530 17635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:45.660481 17635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "7282a94339b7439d95d369e2d35e404d"
format_stamp: "Formatted at 2026-08-06 13:06:45 on dist-test-slave-q9h9"
server_key: "74b7934e86ea9285095b5bd86bab0b7b"
server_key_iv: "e65faadecde6ca0d4625c4fb4ac5dcff"
server_key_version: "encryptionkey@0"
I20260806 13:06:45.660807 17635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "7282a94339b7439d95d369e2d35e404d"
format_stamp: "Formatted at 2026-08-06 13:06:45 on dist-test-slave-q9h9"
server_key: "74b7934e86ea9285095b5bd86bab0b7b"
server_key_iv: "e65faadecde6ca0d4625c4fb4ac5dcff"
server_key_version: "encryptionkey@0"
I20260806 13:06:45.662004 17635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:06:45.662727 17655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:45.662878 17635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:45.662959 17635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "7282a94339b7439d95d369e2d35e404d"
format_stamp: "Formatted at 2026-08-06 13:06:45 on dist-test-slave-q9h9"
server_key: "74b7934e86ea9285095b5bd86bab0b7b"
server_key_iv: "e65faadecde6ca0d4625c4fb4ac5dcff"
server_key_version: "encryptionkey@0"
I20260806 13:06:45.663025 17635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:45.693298 17635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:45.695058 17635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:45.695185 17635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:45.698886 17635 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:33847
I20260806 13:06:45.698927 17734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:33847 every 8 connection(s)
I20260806 13:06:45.699232 17635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:45.700033 17735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:45.700906 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 17635
I20260806 13:06:45.700989 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:45.701136 14209 external_mini_cluster.cc:1428] Setting key 5e9db964acc0b8af237171f241812151
I20260806 13:06:45.702734 17735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d: Bootstrap starting.
I20260806 13:06:45.703328 17735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:45.703640 17735 log.cc:824] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:45.704355 17735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d: No bootstrap required, opened a new log
I20260806 13:06:45.706409 17735 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7282a94339b7439d95d369e2d35e404d" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33847 } }
I20260806 13:06:45.706538 17735 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:45.706571 17735 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7282a94339b7439d95d369e2d35e404d, State: Initialized, Role: FOLLOWER
I20260806 13:06:45.706689 17735 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7282a94339b7439d95d369e2d35e404d" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33847 } }
I20260806 13:06:45.706765 17735 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:45.706796 17735 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:45.706844 17735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:45.707352 17735 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7282a94339b7439d95d369e2d35e404d" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33847 } }
I20260806 13:06:45.707463 17735 leader_election.cc:302] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7282a94339b7439d95d369e2d35e404d; no voters:
I20260806 13:06:45.707625 17735 leader_election.cc:288] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:45.707674 17743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:45.707824 17743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [term 1 LEADER]: Becoming Leader. State: Replica: 7282a94339b7439d95d369e2d35e404d, State: Running, Role: LEADER
I20260806 13:06:45.707870 17735 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 13:06:45 dist-test-slave-q9h9 krb5kdc[17619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:45.707984 17743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7282a94339b7439d95d369e2d35e404d" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33847 } }
I20260806 13:06:45.708307 17743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7282a94339b7439d95d369e2d35e404d. Latest consensus state: current_term: 1 leader_uuid: "7282a94339b7439d95d369e2d35e404d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7282a94339b7439d95d369e2d35e404d" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33847 } } }
I20260806 13:06:45.708454 17743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:45.708531 17746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7282a94339b7439d95d369e2d35e404d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7282a94339b7439d95d369e2d35e404d" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33847 } } }
I20260806 13:06:45.708643 17746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:45.708775 17755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:45.709760 17755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:45.711372 17755 catalog_manager.cc:1347] Generated new cluster ID: 19f0754456654753aa4362e5da1b3ff5
I20260806 13:06:45.711426 17755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:06:45.712059 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:45.701631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35518 (local address 127.13.224.126:33847)
0806 13:06:45.701828 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:45.701838 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:45.701870 (+ 32us) server_negotiation.cc:407] Connection header received
0806 13:06:45.702461 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:45.702470 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:45.702533 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:45.702675 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:45.703310 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:45.704569 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:45.705294 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:45.705906 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:45.706043 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:45.708515 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:45.708520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:45.708521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:45.708533 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:45.710412 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:45.710808 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:45.710811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:45.710812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:45.710914 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:45.711122 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:45.711123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:45.711124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:45.711403 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:45.711548 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:45.711774 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 13:06:45.711903 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
I20260806 13:06:45.721266 17755 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:45.721925 17755 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:45.726297 17755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7282a94339b7439d95d369e2d35e404d: Generated new TSK 0
I20260806 13:06:45.726449 17755 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:06:45.752430 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33847
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:39043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:45.854769 17769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:45.854972 17769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:45.854995 17769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:45.856125 17769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:45.856165 17769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:45.856189 17769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:06:45.857589 17769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:39043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:45.857829 17769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:45.858062 17769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:45.863274 17785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:45.863287 17782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:45.863258 17781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:45.863278 17769 server_base.cc:1034] running on GCE node
I20260806 13:06:45.863660 17769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:45.863880 17769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:45.865029 17769 hybrid_clock.cc:648] HybridClock initialized: now 1786021605865004 us; error 34 us; skew 500 ppm
Aug 06 13:06:45 dist-test-slave-q9h9 krb5kdc[17619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021605, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:45.867460 17769 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:06:45.867861 17769 webserver.cc:460] Webserver started at http://127.13.224.65:40429/ using document root <none> and password file <none>
I20260806 13:06:45.868075 17769 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:45.868119 17769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:45.868249 17769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:45.869233 17769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "7e069aac996b401f8c081f0648aa5971"
format_stamp: "Formatted at 2026-08-06 13:06:45 on dist-test-slave-q9h9"
server_key: "64789f208e6b69e52bf1063e2507d025"
server_key_iv: "4d503e7e235a84c8b5a5f8f837bb7f3f"
server_key_version: "encryptionkey@0"
I20260806 13:06:45.869546 17769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "7e069aac996b401f8c081f0648aa5971"
format_stamp: "Formatted at 2026-08-06 13:06:45 on dist-test-slave-q9h9"
server_key: "64789f208e6b69e52bf1063e2507d025"
server_key_iv: "4d503e7e235a84c8b5a5f8f837bb7f3f"
server_key_version: "encryptionkey@0"
I20260806 13:06:45.870805 17769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:06:45.871438 17794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:45.871577 17769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:45.871659 17769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "7e069aac996b401f8c081f0648aa5971"
format_stamp: "Formatted at 2026-08-06 13:06:45 on dist-test-slave-q9h9"
server_key: "64789f208e6b69e52bf1063e2507d025"
server_key_iv: "4d503e7e235a84c8b5a5f8f837bb7f3f"
server_key_version: "encryptionkey@0"
I20260806 13:06:45.871718 17769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:45.889195 17769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:45.891144 17769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:45.891283 17769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:45.891471 17769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:45.891736 17769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:45.891769 17769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:45.891795 17769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:45.891813 17769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:45.897476 17769 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:35295
I20260806 13:06:45.897541 17970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:35295 every 8 connection(s)
I20260806 13:06:45.897792 17769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
Aug 06 13:06:45 dist-test-slave-q9h9 krb5kdc[17619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021605, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:45.905798 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:45.898457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:60023 (local address 127.13.224.126:33847)
0806 13:06:45.898527 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:45.898529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:45.898681 (+ 152us) server_negotiation.cc:407] Connection header received
0806 13:06:45.899148 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:45.899150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:45.899172 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:45.899221 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:45.900081 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:45.900578 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:45.901525 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:45.901747 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:45.901801 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:45.903693 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:45.903694 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:45.903695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:45.903701 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:45.904796 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:45.905123 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:45.905124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:45.905124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:45.905170 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:45.905345 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:45.905345 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:45.905345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:45.905439 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:45.905483 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:45.905674 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 13:06:45.905736 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 13:06:45.905948 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:45.898394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33847 (local address 127.13.224.65:60023)
0806 13:06:45.898563 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:45.898605 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:45.899052 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:45.899286 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:45.899290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:45.899811 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:45.899989 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:45.899993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:45.900673 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:45.900674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:45.901439 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:45.901441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:45.901820 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:45.901821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:45.901936 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:45.902345 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:45.902353 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:45.903611 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:45.904864 (+ 1253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:45.904869 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:45.904871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:45.905067 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:45.905215 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:45.905216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:45.905216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:45.905302 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:45.905481 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:45.905484 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:45.905584 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 13:06:45.905654 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 13:06:45.905762 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260806 13:06:45.906445 17971 heartbeater.cc:347] Connected to a master server at 127.13.224.126:33847
I20260806 13:06:45.906529 17971 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:45.906662 17971 heartbeater.cc:511] Master 127.13.224.126:33847 requested a full tablet report, sending...
I20260806 13:06:45.907117 17675 ts_manager.cc:194] Registered new tserver with Master: 7e069aac996b401f8c081f0648aa5971 (127.13.224.65:35295)
I20260806 13:06:45.907382 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 17769
I20260806 13:06:45.907467 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:06:45.907629 14209 external_mini_cluster.cc:1428] Setting key 4e52b50aa44143cf01db2c140f2dfa0f
I20260806 13:06:45.907970 17675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:60023
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:06:45.930722 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33847
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:39043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:46.010737 17977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:46.010915 17977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:46.010946 17977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:46.012059 17977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:46.012099 17977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:46.012140 17977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:06:46.013537 17977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:39043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:46.013761 17977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:46.013957 17977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:46.018640 17982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:46.018656 17988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:46.018754 17983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:46.018824 17977 server_base.cc:1034] running on GCE node
I20260806 13:06:46.019097 17977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:46.019304 17977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:46.020430 17977 hybrid_clock.cc:648] HybridClock initialized: now 1786021606020422 us; error 27 us; skew 500 ppm
Aug 06 13:06:46 dist-test-slave-q9h9 krb5kdc[17619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021606, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:46.022696 17977 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:06:46.023073 17977 webserver.cc:460] Webserver started at http://127.13.224.66:45091/ using document root <none> and password file <none>
I20260806 13:06:46.023281 17977 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:46.023326 17977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:46.023449 17977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:46.024417 17977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "d8eac507370446c39508d3d73ba299fb"
format_stamp: "Formatted at 2026-08-06 13:06:46 on dist-test-slave-q9h9"
server_key: "b7ca0e738c65df7295088d154ca79d8f"
server_key_iv: "07e36caef91429db851a129b2c53c5cb"
server_key_version: "encryptionkey@0"
I20260806 13:06:46.024724 17977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "d8eac507370446c39508d3d73ba299fb"
format_stamp: "Formatted at 2026-08-06 13:06:46 on dist-test-slave-q9h9"
server_key: "b7ca0e738c65df7295088d154ca79d8f"
server_key_iv: "07e36caef91429db851a129b2c53c5cb"
server_key_version: "encryptionkey@0"
I20260806 13:06:46.025883 17977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:06:46.026604 17998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:46.026767 17977 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:46.026851 17977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "d8eac507370446c39508d3d73ba299fb"
format_stamp: "Formatted at 2026-08-06 13:06:46 on dist-test-slave-q9h9"
server_key: "b7ca0e738c65df7295088d154ca79d8f"
server_key_iv: "07e36caef91429db851a129b2c53c5cb"
server_key_version: "encryptionkey@0"
I20260806 13:06:46.026903 17977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:46.042713 17977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:46.044678 17977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:46.044785 17977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:46.044958 17977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:46.045190 17977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:46.045219 17977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:46.045261 17977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:46.045277 17977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:46.050745 17977 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:40351
I20260806 13:06:46.050819 18160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:40351 every 8 connection(s)
I20260806 13:06:46.051076 17977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:06:46.054852 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 17977
I20260806 13:06:46.054955 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:06:46.055143 14209 external_mini_cluster.cc:1428] Setting key 9de02459a64ff558bf22a73f668db7a5
Aug 06 13:06:46 dist-test-slave-q9h9 krb5kdc[17619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021606, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:46.060189 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:46.051731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:48957 (local address 127.13.224.126:33847)
0806 13:06:46.051799 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:46.051801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:46.051965 (+ 164us) server_negotiation.cc:407] Connection header received
0806 13:06:46.052455 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:46.052457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:46.052469 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:46.052520 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:46.053401 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:46.053894 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:46.055051 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:46.055287 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:46.055359 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:46.057607 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:46.057608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:46.057609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:46.057616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:46.058947 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:46.059304 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:46.059305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:46.059306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:46.059361 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:46.059589 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:46.059590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:46.059591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:46.059700 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:46.059755 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:46.060010 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 13:06:46.060097 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 13:06:46.060324 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:46.051673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33847 (local address 127.13.224.66:48957)
0806 13:06:46.051853 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:46.051890 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:46.052357 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:46.052586 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:46.052590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:46.053113 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:46.053303 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:46.053308 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:46.053981 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:46.053982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:46.054945 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:46.054949 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:46.055382 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:46.055384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:46.055529 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:46.055996 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:46.056007 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:46.057533 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:46.059015 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:46.059021 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:46.059023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:46.059218 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:46.059420 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:46.059421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:46.059421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:46.059518 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:46.059766 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:46.059773 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:46.059888 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 13:06:46.059975 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 13:06:46.060119 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260806 13:06:46.060835 18163 heartbeater.cc:347] Connected to a master server at 127.13.224.126:33847
I20260806 13:06:46.060921 18163 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:46.061076 18163 heartbeater.cc:511] Master 127.13.224.126:33847 requested a full tablet report, sending...
I20260806 13:06:46.061432 17674 ts_manager.cc:194] Registered new tserver with Master: d8eac507370446c39508d3d73ba299fb (127.13.224.66:40351)
I20260806 13:06:46.061793 17674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:48957
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:06:46.077939 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33847
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:39043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:46.157850 18170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:46.158066 18170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:46.158089 18170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:46.159214 18170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:46.159255 18170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:46.159281 18170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:06:46.160693 18170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:39043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:46.160977 18170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:46.161180 18170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:46.165891 18182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:46.165925 18178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:46.165892 18180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:46.166002 18170 server_base.cc:1034] running on GCE node
I20260806 13:06:46.166396 18170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:46.166626 18170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:46.167761 18170 hybrid_clock.cc:648] HybridClock initialized: now 1786021606167748 us; error 24 us; skew 500 ppm
Aug 06 13:06:46 dist-test-slave-q9h9 krb5kdc[17619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021606, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:46.170082 18170 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:06:46.170480 18170 webserver.cc:460] Webserver started at http://127.13.224.67:42653/ using document root <none> and password file <none>
I20260806 13:06:46.170673 18170 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:46.170720 18170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:46.170838 18170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:46.171788 18170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "bb5b62e9ca1845e7a28dd1a6ca505fa3"
format_stamp: "Formatted at 2026-08-06 13:06:46 on dist-test-slave-q9h9"
server_key: "abe0f7cacb82b9b792569dfb8e167fad"
server_key_iv: "4984d113afcef0c5f1d2464decdb5ec8"
server_key_version: "encryptionkey@0"
I20260806 13:06:46.172124 18170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "bb5b62e9ca1845e7a28dd1a6ca505fa3"
format_stamp: "Formatted at 2026-08-06 13:06:46 on dist-test-slave-q9h9"
server_key: "abe0f7cacb82b9b792569dfb8e167fad"
server_key_iv: "4984d113afcef0c5f1d2464decdb5ec8"
server_key_version: "encryptionkey@0"
I20260806 13:06:46.173300 18170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 13:06:46.173935 18195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:46.174093 18170 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:46.174175 18170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "bb5b62e9ca1845e7a28dd1a6ca505fa3"
format_stamp: "Formatted at 2026-08-06 13:06:46 on dist-test-slave-q9h9"
server_key: "abe0f7cacb82b9b792569dfb8e167fad"
server_key_iv: "4984d113afcef0c5f1d2464decdb5ec8"
server_key_version: "encryptionkey@0"
I20260806 13:06:46.174245 18170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:46.187960 18170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:46.189584 18170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:46.189710 18170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:46.189910 18170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:46.190227 18170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:46.190270 18170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:46.190299 18170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:46.190315 18170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:46.195951 18170 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:33597
I20260806 13:06:46.196013 18360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:33597 every 8 connection(s)
I20260806 13:06:46.196290 18170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
Aug 06 13:06:46 dist-test-slave-q9h9 krb5kdc[17619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021606, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:46.202471 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 18170
I20260806 13:06:46.202561 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:06:46.202752 14209 external_mini_cluster.cc:1428] Setting key 81cadde0e1a8939db87cb7d1a43c5587
I20260806 13:06:46.204496 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:46.196933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:40091 (local address 127.13.224.126:33847)
0806 13:06:46.196981 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:46.196983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:46.197169 (+ 186us) server_negotiation.cc:407] Connection header received
0806 13:06:46.197597 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:46.197598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:46.197609 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:46.197646 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:46.198472 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:46.198995 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:46.199887 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:46.200059 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:46.200099 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:46.201938 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:46.201939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:46.201939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:46.201946 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:46.203212 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:46.203579 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:46.203580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:46.203581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:46.203637 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:46.203872 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:46.203873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:46.203873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:46.203979 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:46.204035 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:46.204331 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 13:06:46.204409 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:06:46.204632 18364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:46.196901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33847 (local address 127.13.224.67:40091)
0806 13:06:46.197060 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:46.197092 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:46.197514 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:46.197696 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:46.197699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:46.198231 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:46.198409 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:46.198413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:46.199060 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:46.199061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:46.199829 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:46.199831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:46.200111 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:46.200111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:46.200207 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:46.200595 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:46.200603 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:46.201861 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:46.203298 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:46.203305 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:46.203307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:46.203509 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:46.203694 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:46.203695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:46.203695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:46.203810 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:46.204040 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:46.204047 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:46.204194 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 13:06:46.204294 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 13:06:46.204436 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260806 13:06:46.205068 18361 heartbeater.cc:347] Connected to a master server at 127.13.224.126:33847
I20260806 13:06:46.205152 18361 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:46.205315 18361 heartbeater.cc:511] Master 127.13.224.126:33847 requested a full tablet report, sending...
I20260806 13:06:46.205648 17674 ts_manager.cc:194] Registered new tserver with Master: bb5b62e9ca1845e7a28dd1a6ca505fa3 (127.13.224.67:33597)
I20260806 13:06:46.206000 17674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:40091
I20260806 13:06:46.214931 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 13:06:46.227097 17738 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:06:46.226192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35520 (local address 127.13.224.126:33847)
0806 13:06:46.226249 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:46.226251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:46.226382 (+ 131us) server_negotiation.cc:407] Connection header received
0806 13:06:46.226461 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:46.226462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:46.226473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:46.226526 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:46.227037 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35520: BlockingRecv error: recv got EOF from 127.0.0.1:35520 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:06:46.227841 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 17769
I20260806 13:06:46.244762 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 17977
I20260806 13:06:46.251822 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 18170
I20260806 13:06:46.262863 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 17635
2026-08-06T13:06:46Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1929 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:06:46 dist-test-slave-q9h9 krb5kdc[18383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:06:46 dist-test-slave-q9h9 krb5kdc[18383](info): set up 2 sockets
Aug 06 13:06:46 dist-test-slave-q9h9 krb5kdc[18383](info): commencing operation
krb5kdc: starting...
W20260806 13:06:47.361054 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:06:47 dist-test-slave-q9h9 krb5kdc[18383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:06:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:06:47.469563 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:40857
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:37957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:40857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:47.547544 18402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:47.547732 18402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:47.547752 18402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:47.548827 18402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:47.548861 18402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:47.548872 18402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:47.548888 18402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:47.548902 18402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:06:47.550326 18402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:37957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:40857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:40857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:47.550499 18402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:47.550664 18402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:47.554988 18414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:47.555016 18409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:47.555020 18412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:47.555052 18402 server_base.cc:1034] running on GCE node
I20260806 13:06:47.555400 18402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:47.555611 18402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:47.556759 18402 hybrid_clock.cc:648] HybridClock initialized: now 1786021607556746 us; error 29 us; skew 500 ppm
Aug 06 13:06:47 dist-test-slave-q9h9 krb5kdc[18383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021607, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:47.559051 18402 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:06:47.559410 18402 webserver.cc:460] Webserver started at http://127.13.224.126:33273/ using document root <none> and password file <none>
I20260806 13:06:47.559587 18402 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:47.559623 18402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:47.559708 18402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:47.560590 18402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "c21a54365bdb44dc8d18be17b838df8b"
format_stamp: "Formatted at 2026-08-06 13:06:47 on dist-test-slave-q9h9"
server_key: "d447a6fdfbd971318692b25c1584b572"
server_key_iv: "10ec62bedeb202c85eaeeed92fa172d1"
server_key_version: "encryptionkey@0"
I20260806 13:06:47.560864 18402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "c21a54365bdb44dc8d18be17b838df8b"
format_stamp: "Formatted at 2026-08-06 13:06:47 on dist-test-slave-q9h9"
server_key: "d447a6fdfbd971318692b25c1584b572"
server_key_iv: "10ec62bedeb202c85eaeeed92fa172d1"
server_key_version: "encryptionkey@0"
I20260806 13:06:47.561977 18402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:06:47.562671 18423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:47.562815 18402 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:47.562887 18402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "c21a54365bdb44dc8d18be17b838df8b"
format_stamp: "Formatted at 2026-08-06 13:06:47 on dist-test-slave-q9h9"
server_key: "d447a6fdfbd971318692b25c1584b572"
server_key_iv: "10ec62bedeb202c85eaeeed92fa172d1"
server_key_version: "encryptionkey@0"
I20260806 13:06:47.562947 18402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:47.599514 18402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:47.601100 18402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:47.601238 18402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:47.604952 18402 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:40857
I20260806 13:06:47.604985 18492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:40857 every 8 connection(s)
I20260806 13:06:47.605491 18402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:47.606169 18493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:47.608300 18493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b: Bootstrap starting.
I20260806 13:06:47.608820 18493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:47.609045 18493 log.cc:824] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:47.609673 18493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b: No bootstrap required, opened a new log
I20260806 13:06:47.611158 18493 raft_consensus.cc:348] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c21a54365bdb44dc8d18be17b838df8b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 40857 } }
I20260806 13:06:47.611300 18493 raft_consensus.cc:374] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:47.611331 18493 raft_consensus.cc:729] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c21a54365bdb44dc8d18be17b838df8b, State: Initialized, Role: FOLLOWER
I20260806 13:06:47.611435 18493 consensus_queue.cc:260] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c21a54365bdb44dc8d18be17b838df8b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 40857 } }
I20260806 13:06:47.611511 18493 raft_consensus.cc:388] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:47.611543 18493 raft_consensus.cc:482] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:47.611575 18493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:47.612365 18493 raft_consensus.cc:504] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c21a54365bdb44dc8d18be17b838df8b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 40857 } }
I20260806 13:06:47.612452 18493 leader_election.cc:302] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c21a54365bdb44dc8d18be17b838df8b; no voters:
I20260806 13:06:47.612571 18493 leader_election.cc:288] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:47.612627 18496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:47.612782 18493 sys_catalog.cc:564] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:47.612816 18496 raft_consensus.cc:686] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [term 1 LEADER]: Becoming Leader. State: Replica: c21a54365bdb44dc8d18be17b838df8b, State: Running, Role: LEADER
I20260806 13:06:47.612957 18496 consensus_queue.cc:237] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c21a54365bdb44dc8d18be17b838df8b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 40857 } }
I20260806 13:06:47.613345 18499 sys_catalog.cc:455] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c21a54365bdb44dc8d18be17b838df8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c21a54365bdb44dc8d18be17b838df8b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 40857 } } }
I20260806 13:06:47.613426 18499 sys_catalog.cc:458] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:47.613591 18500 sys_catalog.cc:455] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c21a54365bdb44dc8d18be17b838df8b. Latest consensus state: current_term: 1 leader_uuid: "c21a54365bdb44dc8d18be17b838df8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c21a54365bdb44dc8d18be17b838df8b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 40857 } } }
I20260806 13:06:47.613626 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 18402
I20260806 13:06:47.613701 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:47.613683 18510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:47.613724 18500 sys_catalog.cc:458] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:47.613897 14209 external_mini_cluster.cc:1428] Setting key fe6d8cd7d1f35b1bacb898763fae9f58
I20260806 13:06:47.614214 18510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:47.615922 18510 catalog_manager.cc:1347] Generated new cluster ID: 9a056234d1e749a086c3dc8093879ea3
I20260806 13:06:47.615980 18510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:06:47 dist-test-slave-q9h9 krb5kdc[18383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:47.622968 18520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:47.614454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.13.224.126:40857)
0806 13:06:47.614886 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:47.614895 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:47.614927 (+ 32us) server_negotiation.cc:407] Connection header received
0806 13:06:47.615037 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:47.615044 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:47.615095 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:47.615220 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:47.615896 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:47.616836 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:47.617548 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:47.617975 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:47.618092 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:47.620167 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:47.620171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:47.620171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:47.620181 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:47.621607 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:47.621925 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:47.621928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:47.621928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:47.622041 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:47.622223 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:47.622224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:47.622224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:47.622441 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:47.622578 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:47.622680 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 13:06:47.622816 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":71,"threads_started":1}
I20260806 13:06:47.623787 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:06:47.642403 18510 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:47.643100 18510 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:47.657856 18520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:47.652281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.13.224.126:40857)
0806 13:06:47.652328 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:47.652330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:47.652807 (+ 477us) server_negotiation.cc:407] Connection header received
0806 13:06:47.652980 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:47.652981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:47.652993 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:47.653144 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:47.653669 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:47.655034 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:47.656112 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:47.656440 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:47.656495 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:47.657225 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:47.657227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:47.657227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:47.657450 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:47.657469 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:47.657478 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:47.657527 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:47.657757 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 13:06:47.657779 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:06:47.662786 18510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c21a54365bdb44dc8d18be17b838df8b: Generated new TSK 0
I20260806 13:06:47.662947 18510 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:06:47.669256 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 18402
2026-08-06T13:06:47Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1404 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:06:47 dist-test-slave-q9h9 krb5kdc[18536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:06:47 dist-test-slave-q9h9 krb5kdc[18536](info): set up 2 sockets
Aug 06 13:06:47 dist-test-slave-q9h9 krb5kdc[18536](info): commencing operation
krb5kdc: starting...
W20260806 13:06:48.782166 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:06:48 dist-test-slave-q9h9 krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:06:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:06:48.886899 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:43217
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:43481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:43217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:48.965726 18558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:48.965911 18558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:48.965941 18558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:48.967067 18558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:48.967109 18558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:48.967129 18558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:48.967145 18558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:48.967160 18558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:06:48.968609 18558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:43481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:43217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:43217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:48.968890 18558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:48.969168 18558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:48.973760 18569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:48.973768 18567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:48.973829 18564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:48.973847 18558 server_base.cc:1034] running on GCE node
I20260806 13:06:48.974232 18558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:48.974480 18558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:48.975620 18558 hybrid_clock.cc:648] HybridClock initialized: now 1786021608975607 us; error 30 us; skew 500 ppm
Aug 06 13:06:48 dist-test-slave-q9h9 krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021608, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:48.977800 18558 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:06:48.978232 18558 webserver.cc:460] Webserver started at http://127.13.224.126:43375/ using document root <none> and password file <none>
I20260806 13:06:48.978425 18558 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:48.978462 18558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:48.978552 18558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:48.979470 18558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "51abf75704f849a1a25afa297ffd2ed8"
format_stamp: "Formatted at 2026-08-06 13:06:48 on dist-test-slave-q9h9"
server_key: "f39502232d1b0d002b90cfcaa81ac092"
server_key_iv: "07355802328bbd277388a8e47c3740a7"
server_key_version: "encryptionkey@0"
I20260806 13:06:48.979748 18558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "51abf75704f849a1a25afa297ffd2ed8"
format_stamp: "Formatted at 2026-08-06 13:06:48 on dist-test-slave-q9h9"
server_key: "f39502232d1b0d002b90cfcaa81ac092"
server_key_iv: "07355802328bbd277388a8e47c3740a7"
server_key_version: "encryptionkey@0"
I20260806 13:06:48.980875 18558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:06:48.981480 18578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:48.981626 18558 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:48.981699 18558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "51abf75704f849a1a25afa297ffd2ed8"
format_stamp: "Formatted at 2026-08-06 13:06:48 on dist-test-slave-q9h9"
server_key: "f39502232d1b0d002b90cfcaa81ac092"
server_key_iv: "07355802328bbd277388a8e47c3740a7"
server_key_version: "encryptionkey@0"
I20260806 13:06:48.981763 18558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:48.990284 18558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:48.992198 18558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:48.992338 18558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:48.995919 18558 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:43217
I20260806 13:06:48.995954 18649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:43217 every 8 connection(s)
I20260806 13:06:48.996227 18558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:48.997321 18650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:48.999377 18650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8: Bootstrap starting.
I20260806 13:06:48.999871 18650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:49.000087 18650 log.cc:824] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:49.000737 18650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8: No bootstrap required, opened a new log
I20260806 13:06:49.000752 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 18558
I20260806 13:06:49.000852 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:49.001026 14209 external_mini_cluster.cc:1428] Setting key d9bf28090731272a01bae5e08230eab8
I20260806 13:06:49.002568 18650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51abf75704f849a1a25afa297ffd2ed8" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43217 } }
I20260806 13:06:49.002710 18650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:49.002746 18650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51abf75704f849a1a25afa297ffd2ed8, State: Initialized, Role: FOLLOWER
I20260806 13:06:49.002872 18650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51abf75704f849a1a25afa297ffd2ed8" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43217 } }
I20260806 13:06:49.002950 18650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:49.002986 18650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:49.003031 18650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:49.003922 18650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51abf75704f849a1a25afa297ffd2ed8" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43217 } }
I20260806 13:06:49.004019 18650 leader_election.cc:302] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51abf75704f849a1a25afa297ffd2ed8; no voters:
I20260806 13:06:49.004171 18650 leader_election.cc:288] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:49.004207 18655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:49.004310 18655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [term 1 LEADER]: Becoming Leader. State: Replica: 51abf75704f849a1a25afa297ffd2ed8, State: Running, Role: LEADER
I20260806 13:06:49.004391 18655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51abf75704f849a1a25afa297ffd2ed8" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43217 } }
I20260806 13:06:49.004439 18650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:49.004679 18655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51abf75704f849a1a25afa297ffd2ed8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51abf75704f849a1a25afa297ffd2ed8" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43217 } } }
I20260806 13:06:49.004758 18655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:49.004874 18655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51abf75704f849a1a25afa297ffd2ed8. Latest consensus state: current_term: 1 leader_uuid: "51abf75704f849a1a25afa297ffd2ed8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51abf75704f849a1a25afa297ffd2ed8" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43217 } } }
I20260806 13:06:49.004931 18655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:49.005071 18663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:49.005558 18663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 13:06:49 dist-test-slave-q9h9 krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:49.007778 18663 catalog_manager.cc:1347] Generated new cluster ID: 168d85a325524f48abc410e964581977
I20260806 13:06:49.007831 18663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:06:49.011080 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:49.001511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50736 (local address 127.13.224.126:43217)
0806 13:06:49.001658 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:49.001665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:49.001688 (+ 23us) server_negotiation.cc:407] Connection header received
0806 13:06:49.002539 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:49.002545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:49.002588 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:49.002690 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:49.003450 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.004409 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.005334 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.005756 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.006180 (+ 424us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:49.008157 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:49.008161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:49.008161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:49.008172 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:49.009638 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:49.009955 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:49.009957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:49.009958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:49.010098 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:49.010321 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:49.010322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:49.010322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:49.010530 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:49.010666 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:49.010770 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 13:06:49.010897 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
I20260806 13:06:49.021343 18663 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:49.021807 18663 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:49.026942 18663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51abf75704f849a1a25afa297ffd2ed8: Generated new TSK 0
I20260806 13:06:49.027094 18663 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:06:49.056571 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43217
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:43481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:49.135607 18683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:49.135794 18683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:49.135825 18683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:49.136926 18683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:49.136965 18683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:49.136999 18683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:06:49.138538 18683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:43481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:49.138767 18683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:49.138971 18683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:49.143615 18695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:49.143630 18693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:49.143821 18683 server_base.cc:1034] running on GCE node
W20260806 13:06:49.143651 18691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:49.144076 18683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:49.144311 18683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:49.145457 18683 hybrid_clock.cc:648] HybridClock initialized: now 1786021609145446 us; error 30 us; skew 500 ppm
Aug 06 13:06:49 dist-test-slave-q9h9 krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021609, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:49.147869 18683 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:06:49.148350 18683 webserver.cc:460] Webserver started at http://127.13.224.65:42343/ using document root <none> and password file <none>
I20260806 13:06:49.148525 18683 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:49.148572 18683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:49.148665 18683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:49.149612 18683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "b9a71849d71e4e71a3721f84ebee675b"
format_stamp: "Formatted at 2026-08-06 13:06:49 on dist-test-slave-q9h9"
server_key: "5b3244262cc4ea1f098f76661d97e2d1"
server_key_iv: "a343c77268dc4d39fe5d3e4db79cacbd"
server_key_version: "encryptionkey@0"
I20260806 13:06:49.149904 18683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "b9a71849d71e4e71a3721f84ebee675b"
format_stamp: "Formatted at 2026-08-06 13:06:49 on dist-test-slave-q9h9"
server_key: "5b3244262cc4ea1f098f76661d97e2d1"
server_key_iv: "a343c77268dc4d39fe5d3e4db79cacbd"
server_key_version: "encryptionkey@0"
I20260806 13:06:49.151289 18683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:49.151988 18706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:49.152168 18683 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:49.152274 18683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "b9a71849d71e4e71a3721f84ebee675b"
format_stamp: "Formatted at 2026-08-06 13:06:49 on dist-test-slave-q9h9"
server_key: "5b3244262cc4ea1f098f76661d97e2d1"
server_key_iv: "a343c77268dc4d39fe5d3e4db79cacbd"
server_key_version: "encryptionkey@0"
I20260806 13:06:49.152330 18683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:49.196038 18683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:49.198129 18683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:49.198242 18683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:49.198434 18683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:49.198720 18683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:49.198752 18683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:49.198781 18683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:49.198804 18683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:49.204589 18683 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:36937
I20260806 13:06:49.204655 18871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:36937 every 8 connection(s)
I20260806 13:06:49.204936 18683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
Aug 06 13:06:49 dist-test-slave-q9h9 krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021609, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:49.211308 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 18683
I20260806 13:06:49.211390 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:06:49.211541 14209 external_mini_cluster.cc:1428] Setting key 71186e0c06eec03523a55c4c37bdc8fb
I20260806 13:06:49.214178 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:49.205601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:60481 (local address 127.13.224.126:43217)
0806 13:06:49.205677 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:49.205679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:49.205873 (+ 194us) server_negotiation.cc:407] Connection header received
0806 13:06:49.206449 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:49.206450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:49.206465 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:49.206514 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:49.207390 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.207877 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.208826 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.209026 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.209075 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:49.210928 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:49.210930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:49.210930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:49.210937 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:49.212401 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:49.212884 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:49.212885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:49.212886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:49.212951 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:49.213227 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:49.213229 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:49.213229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:49.213378 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:49.213444 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:49.213843 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 13:06:49.214064 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 13:06:49.214205 18876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:49.205588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:43217 (local address 127.13.224.65:60481)
0806 13:06:49.205758 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:49.205794 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:49.206365 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:49.206568 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:49.206571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:49.207111 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:49.207303 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:49.207308 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.207961 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.207962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:49.208755 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:49.208758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.209086 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.209087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:49.209183 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:49.209563 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:49.209571 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:49.210841 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:49.212471 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:49.212478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:49.212481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:49.212781 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:49.213025 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:49.213026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:49.213026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:49.213163 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:49.213467 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:49.213475 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:49.213676 (+ 201us) client_negotiation.cc:769] Sending connection context
0806 13:06:49.213791 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 13:06:49.213956 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260806 13:06:49.214747 18872 heartbeater.cc:347] Connected to a master server at 127.13.224.126:43217
I20260806 13:06:49.214854 18872 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:49.215037 18872 heartbeater.cc:511] Master 127.13.224.126:43217 requested a full tablet report, sending...
I20260806 13:06:49.215574 18600 ts_manager.cc:194] Registered new tserver with Master: b9a71849d71e4e71a3721f84ebee675b (127.13.224.65:36937)
I20260806 13:06:49.216531 18600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:60481
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:06:49.238364 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43217
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:43481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:49.318945 18880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:49.319128 18880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:49.319149 18880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:49.320283 18880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:49.320323 18880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:49.320353 18880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:06:49.322118 18880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:43481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:49.322359 18880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:49.322571 18880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:49.327168 18886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:49.327199 18885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:49.327343 18880 server_base.cc:1034] running on GCE node
W20260806 13:06:49.327134 18888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:49.327586 18880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:49.327818 18880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:49.328958 18880 hybrid_clock.cc:648] HybridClock initialized: now 1786021609328941 us; error 37 us; skew 500 ppm
Aug 06 13:06:49 dist-test-slave-q9h9 krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021609, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:49.331315 18880 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:06:49.331746 18880 webserver.cc:460] Webserver started at http://127.13.224.66:43427/ using document root <none> and password file <none>
I20260806 13:06:49.331960 18880 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:49.332007 18880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:49.332124 18880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:49.333062 18880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "4054eec351cb4e0ab308a393646d146a"
format_stamp: "Formatted at 2026-08-06 13:06:49 on dist-test-slave-q9h9"
server_key: "381cffd50dbbf33f658a18183c1f71d4"
server_key_iv: "35dfcf651108753d21284026daaa3586"
server_key_version: "encryptionkey@0"
I20260806 13:06:49.333385 18880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "4054eec351cb4e0ab308a393646d146a"
format_stamp: "Formatted at 2026-08-06 13:06:49 on dist-test-slave-q9h9"
server_key: "381cffd50dbbf33f658a18183c1f71d4"
server_key_iv: "35dfcf651108753d21284026daaa3586"
server_key_version: "encryptionkey@0"
I20260806 13:06:49.334610 18880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:06:49.335248 18899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:49.335395 18880 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:49.335475 18880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "4054eec351cb4e0ab308a393646d146a"
format_stamp: "Formatted at 2026-08-06 13:06:49 on dist-test-slave-q9h9"
server_key: "381cffd50dbbf33f658a18183c1f71d4"
server_key_iv: "35dfcf651108753d21284026daaa3586"
server_key_version: "encryptionkey@0"
I20260806 13:06:49.335531 18880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:49.348255 18880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:49.349812 18880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:49.349933 18880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:49.350139 18880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:49.350407 18880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:49.350440 18880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:49.350466 18880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:49.350490 18880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:49.356032 18880 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:40575
I20260806 13:06:49.356091 19066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:40575 every 8 connection(s)
I20260806 13:06:49.356355 18880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
Aug 06 13:06:49 dist-test-slave-q9h9 krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021609, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:49.362727 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 18880
I20260806 13:06:49.362835 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:06:49.363032 14209 external_mini_cluster.cc:1428] Setting key 1236d5ff2791d9154fa0323216355bfe
I20260806 13:06:49.364912 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:49.356964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:34321 (local address 127.13.224.126:43217)
0806 13:06:49.357018 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:49.357020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:49.357181 (+ 161us) server_negotiation.cc:407] Connection header received
0806 13:06:49.357660 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:49.357662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:49.357675 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:49.357722 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:49.358665 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.359155 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.360119 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.360331 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.360388 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:49.362312 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:49.362314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:49.362314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:49.362322 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:49.363623 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:49.364063 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:49.364063 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:49.364064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:49.364119 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:49.364363 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:49.364364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:49.364364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:49.364467 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:49.364518 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:49.364784 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 13:06:49.364847 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 13:06:49.365124 19070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:49.356928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:43217 (local address 127.13.224.66:34321)
0806 13:06:49.357068 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:49.357110 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:49.357567 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:49.357812 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:49.357816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:49.358386 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:49.358566 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:49.358571 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.359237 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.359238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:49.360029 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:49.360031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.360400 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.360401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:49.360515 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:49.360944 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:49.360952 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:49.362223 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:49.363703 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:49.363710 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:49.363712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:49.363991 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:49.364176 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:49.364177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:49.364178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:49.364306 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:49.364519 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:49.364524 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:49.364667 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 13:06:49.364774 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 13:06:49.364925 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
I20260806 13:06:49.365605 19067 heartbeater.cc:347] Connected to a master server at 127.13.224.126:43217
I20260806 13:06:49.365693 19067 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:49.365854 19067 heartbeater.cc:511] Master 127.13.224.126:43217 requested a full tablet report, sending...
I20260806 13:06:49.366227 18600 ts_manager.cc:194] Registered new tserver with Master: 4054eec351cb4e0ab308a393646d146a (127.13.224.66:40575)
I20260806 13:06:49.366698 18600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:34321
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:06:49.386240 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43217
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:43481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:49.464931 19073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:49.465108 19073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:49.465130 19073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:49.466220 19073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:49.466266 19073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:49.466291 19073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:06:49.467676 19073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:43481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:49.467856 19073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:49.468029 19073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:49.472658 19084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:49.472668 19085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:49.472685 19087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:49.472775 19073 server_base.cc:1034] running on GCE node
I20260806 13:06:49.473031 19073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:49.473242 19073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:49.474378 19073 hybrid_clock.cc:648] HybridClock initialized: now 1786021609474366 us; error 28 us; skew 500 ppm
Aug 06 13:06:49 dist-test-slave-q9h9 krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021609, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:49.480638 19073 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:06:49.481033 19073 webserver.cc:460] Webserver started at http://127.13.224.67:34905/ using document root <none> and password file <none>
I20260806 13:06:49.481230 19073 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:49.481273 19073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:49.481424 19073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:49.482417 19073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "b6ac13fe2c9f471fb058898973f728d6"
format_stamp: "Formatted at 2026-08-06 13:06:49 on dist-test-slave-q9h9"
server_key: "3306bf95087985db44d922048d115b05"
server_key_iv: "f9d0b90e4d86df77264df04d7bfa257f"
server_key_version: "encryptionkey@0"
I20260806 13:06:49.482731 19073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "b6ac13fe2c9f471fb058898973f728d6"
format_stamp: "Formatted at 2026-08-06 13:06:49 on dist-test-slave-q9h9"
server_key: "3306bf95087985db44d922048d115b05"
server_key_iv: "f9d0b90e4d86df77264df04d7bfa257f"
server_key_version: "encryptionkey@0"
I20260806 13:06:49.483923 19073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:06:49.484616 19096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:49.484783 19073 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:49.484862 19073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "b6ac13fe2c9f471fb058898973f728d6"
format_stamp: "Formatted at 2026-08-06 13:06:49 on dist-test-slave-q9h9"
server_key: "3306bf95087985db44d922048d115b05"
server_key_iv: "f9d0b90e4d86df77264df04d7bfa257f"
server_key_version: "encryptionkey@0"
I20260806 13:06:49.484915 19073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:49.512422 19073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:49.514350 19073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:49.514454 19073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:49.514622 19073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:49.514889 19073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:49.514927 19073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:49.514946 19073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:49.514963 19073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:49.520494 19073 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:37325
I20260806 13:06:49.520552 19251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:37325 every 8 connection(s)
I20260806 13:06:49.520813 19073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
Aug 06 13:06:49 dist-test-slave-q9h9 krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021609, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:49.528590 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:49.521405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:47647 (local address 127.13.224.126:43217)
0806 13:06:49.521445 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:49.521448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:49.521623 (+ 175us) server_negotiation.cc:407] Connection header received
0806 13:06:49.522103 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:49.522104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:49.522116 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:49.522163 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:49.522974 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.523451 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.524375 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.524564 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.524617 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:49.526416 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:49.526418 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:49.526418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:49.526425 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:49.527606 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:49.527942 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:49.527943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:49.527943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:49.527987 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:49.528154 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:49.528155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:49.528155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:49.528242 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:49.528284 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:49.528472 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 13:06:49.528527 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:06:49.528740 19254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:49.521382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:43217 (local address 127.13.224.67:47647)
0806 13:06:49.521525 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:49.521558 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:49.521984 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:49.522215 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:49.522218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:49.522738 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:49.522909 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:49.522913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.523510 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.523511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:49.524304 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:49.524307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.524624 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.524625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:49.524718 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:49.525055 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:49.525063 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:49.526328 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:49.527673 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:49.527679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:49.527681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:49.527887 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:49.528030 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:49.528031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:49.528031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:49.528114 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:49.528282 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:49.528284 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:49.528382 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 13:06:49.528454 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 13:06:49.528558 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260806 13:06:49.529170 19252 heartbeater.cc:347] Connected to a master server at 127.13.224.126:43217
I20260806 13:06:49.529241 19252 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:49.529389 19252 heartbeater.cc:511] Master 127.13.224.126:43217 requested a full tablet report, sending...
I20260806 13:06:49.529706 18600 ts_manager.cc:194] Registered new tserver with Master: b6ac13fe2c9f471fb058898973f728d6 (127.13.224.67:37325)
I20260806 13:06:49.530118 18600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:47647
I20260806 13:06:49.530439 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 19073
I20260806 13:06:49.530527 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:06:49.530704 14209 external_mini_cluster.cc:1428] Setting key 192c95bf2253aff16ef3082ea73b712f
I20260806 13:06:49.531417 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 13:06:49 dist-test-slave-q9h9 krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021609, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 13:06:49 dist-test-slave-q9h9 krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021609, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:49.552690 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:49.546577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50752 (local address 127.13.224.126:43217)
0806 13:06:49.546634 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:49.546636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:49.546723 (+ 87us) server_negotiation.cc:407] Connection header received
0806 13:06:49.546785 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:49.546786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:49.546795 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:49.546837 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:49.547384 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.547835 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.548531 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:49.548710 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:49.548766 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:49.550715 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:49.550716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:49.550717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:49.550723 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:49.551716 (+ 993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:49.552010 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:49.552011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:49.552012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:49.552061 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:49.552217 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:49.552218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:49.552218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:49.552350 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:49.552435 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:49.552535 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 13:06:49.552605 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 13:06:49.552968 18600 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50752
I20260806 13:06:49.553287 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 18683
I20260806 13:06:49.569698 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 18880
I20260806 13:06:49.577934 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 19073
I20260806 13:06:49.585691 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 18558
2026-08-06T13:06:49Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1915 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:06:49 dist-test-slave-q9h9 krb5kdc[19265](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:06:49 dist-test-slave-q9h9 krb5kdc[19265](info): set up 2 sockets
Aug 06 13:06:49 dist-test-slave-q9h9 krb5kdc[19265](info): commencing operation
krb5kdc: starting...
W20260806 13:06:50.688813 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:06:50 dist-test-slave-q9h9 krb5kdc[19265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:06:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:06:50.794667 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:43235
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:43235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:50.872820 19284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:50.873008 19284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:50.873039 19284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:50.874171 19284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:50.874207 19284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:50.874235 19284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:50.874251 19284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:50.874277 19284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:06:50.875762 19284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:43235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:43235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:50.875993 19284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:50.876179 19284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:50.880578 19291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:50.880638 19292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:50.880638 19294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:50.880779 19284 server_base.cc:1034] running on GCE node
I20260806 13:06:50.881013 19284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:50.881271 19284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:50.882403 19284 hybrid_clock.cc:648] HybridClock initialized: now 1786021610882389 us; error 28 us; skew 500 ppm
Aug 06 13:06:50 dist-test-slave-q9h9 krb5kdc[19265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021610, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:50.884598 19284 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:06:50.884987 19284 webserver.cc:460] Webserver started at http://127.13.224.126:44057/ using document root <none> and password file <none>
I20260806 13:06:50.885169 19284 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:50.885213 19284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:50.885335 19284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:50.886332 19284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "1ea0665ab5fc48b489d71a9e5bd8e158"
format_stamp: "Formatted at 2026-08-06 13:06:50 on dist-test-slave-q9h9"
server_key: "934a0772a0779258997d9d69780013e5"
server_key_iv: "4b3bf614cf807cda490e647ad7b5b130"
server_key_version: "encryptionkey@0"
I20260806 13:06:50.886634 19284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "1ea0665ab5fc48b489d71a9e5bd8e158"
format_stamp: "Formatted at 2026-08-06 13:06:50 on dist-test-slave-q9h9"
server_key: "934a0772a0779258997d9d69780013e5"
server_key_iv: "4b3bf614cf807cda490e647ad7b5b130"
server_key_version: "encryptionkey@0"
I20260806 13:06:50.887825 19284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:06:50.888448 19302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:50.888600 19284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:50.888690 19284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "1ea0665ab5fc48b489d71a9e5bd8e158"
format_stamp: "Formatted at 2026-08-06 13:06:50 on dist-test-slave-q9h9"
server_key: "934a0772a0779258997d9d69780013e5"
server_key_iv: "4b3bf614cf807cda490e647ad7b5b130"
server_key_version: "encryptionkey@0"
I20260806 13:06:50.888758 19284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:50.900288 19284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:50.901773 19284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:50.901889 19284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:50.905459 19284 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:43235
I20260806 13:06:50.905489 19379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:43235 every 8 connection(s)
I20260806 13:06:50.905920 19284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:50.906584 19380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:50.908309 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 19284
I20260806 13:06:50.908394 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:50.908552 14209 external_mini_cluster.cc:1428] Setting key b9602d588a5db872b357b743522a39cf
I20260806 13:06:50.908732 19380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158: Bootstrap starting.
I20260806 13:06:50.909443 19380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:50.909771 19380 log.cc:824] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:50.910457 19380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158: No bootstrap required, opened a new log
I20260806 13:06:50.912174 19380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea0665ab5fc48b489d71a9e5bd8e158" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43235 } }
I20260806 13:06:50.912331 19380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:50.912375 19380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ea0665ab5fc48b489d71a9e5bd8e158, State: Initialized, Role: FOLLOWER
I20260806 13:06:50.912515 19380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea0665ab5fc48b489d71a9e5bd8e158" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43235 } }
I20260806 13:06:50.912592 19380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:50.912631 19380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:50.912678 19380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:50.913336 19380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea0665ab5fc48b489d71a9e5bd8e158" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43235 } }
I20260806 13:06:50.913445 19380 leader_election.cc:302] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ea0665ab5fc48b489d71a9e5bd8e158; no voters:
I20260806 13:06:50.913597 19380 leader_election.cc:288] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:50.913641 19386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:50.913776 19386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [term 1 LEADER]: Becoming Leader. State: Replica: 1ea0665ab5fc48b489d71a9e5bd8e158, State: Running, Role: LEADER
I20260806 13:06:50.913853 19380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:50.913892 19386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea0665ab5fc48b489d71a9e5bd8e158" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43235 } }
I20260806 13:06:50.914256 19388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ea0665ab5fc48b489d71a9e5bd8e158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea0665ab5fc48b489d71a9e5bd8e158" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43235 } } }
I20260806 13:06:50.914336 19388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:50.914503 19389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ea0665ab5fc48b489d71a9e5bd8e158. Latest consensus state: current_term: 1 leader_uuid: "1ea0665ab5fc48b489d71a9e5bd8e158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea0665ab5fc48b489d71a9e5bd8e158" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43235 } } }
I20260806 13:06:50.914562 19389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158 [sys.catalog]: This master's current role is: LEADER
Aug 06 13:06:50 dist-test-slave-q9h9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
W20260806 13:06:50.915606 19411 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:06:50.915649 19411 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:06:50.915692 19395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:50.915835 19395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:50.917495 19395 catalog_manager.cc:1347] Generated new cluster ID: bbb804952c2545958ad088bb0e83a9ca
I20260806 13:06:50.917542 19395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:06:50.919369 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:50.909001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59942 (local address 127.13.224.126:43235)
0806 13:06:50.909189 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:50.909198 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:50.909237 (+ 39us) server_negotiation.cc:407] Connection header received
0806 13:06:50.909774 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:50.909782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:50.909841 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:50.909987 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:50.910668 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:50.911504 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:50.912216 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:50.912971 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:50.913077 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:50.916111 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:50.916115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:50.916115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:50.916126 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:50.917858 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:50.918273 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:50.918275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:50.918276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:50.918368 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:50.918562 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:50.918562 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:50.918563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:50.918821 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:50.918945 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:50.919065 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 13:06:50.919181 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260806 13:06:50.937848 19395 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:50.938623 19395 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:50.943156 19395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ea0665ab5fc48b489d71a9e5bd8e158: Generated new TSK 0
I20260806 13:06:50.943318 19395 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:06:50.968528 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43235
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:51.047654 19416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:51.047850 19416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:51.047880 19416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:51.048993 19416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:51.049033 19416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:51.049067 19416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:06:51.050551 19416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:51.050781 19416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:51.050989 19416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:51.055958 19426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:51.055981 19424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:51.056077 19416 server_base.cc:1034] running on GCE node
W20260806 13:06:51.055958 19423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:51.056409 19416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:51.056639 19416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:51.057789 19416 hybrid_clock.cc:648] HybridClock initialized: now 1786021611057770 us; error 31 us; skew 500 ppm
Aug 06 13:06:51 dist-test-slave-q9h9 krb5kdc[19265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021611, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:51.060256 19416 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:06:51.060670 19416 webserver.cc:460] Webserver started at http://127.13.224.65:43479/ using document root <none> and password file <none>
I20260806 13:06:51.060865 19416 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:51.060914 19416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:51.061039 19416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:51.062127 19416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "8ee0575513254f53af53d66624949076"
format_stamp: "Formatted at 2026-08-06 13:06:51 on dist-test-slave-q9h9"
server_key: "bb09fab1a4e85f9661c0fc2dc1965ffe"
server_key_iv: "3f9494a9e0734d15d37b127ec100c25d"
server_key_version: "encryptionkey@0"
I20260806 13:06:51.062456 19416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "8ee0575513254f53af53d66624949076"
format_stamp: "Formatted at 2026-08-06 13:06:51 on dist-test-slave-q9h9"
server_key: "bb09fab1a4e85f9661c0fc2dc1965ffe"
server_key_iv: "3f9494a9e0734d15d37b127ec100c25d"
server_key_version: "encryptionkey@0"
I20260806 13:06:51.063673 19416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:51.064312 19435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:51.064459 19416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:51.064549 19416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "8ee0575513254f53af53d66624949076"
format_stamp: "Formatted at 2026-08-06 13:06:51 on dist-test-slave-q9h9"
server_key: "bb09fab1a4e85f9661c0fc2dc1965ffe"
server_key_iv: "3f9494a9e0734d15d37b127ec100c25d"
server_key_version: "encryptionkey@0"
I20260806 13:06:51.064611 19416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:51.081337 19416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:51.083009 19416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:51.083135 19416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:51.083348 19416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:51.083635 19416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:51.083670 19416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:51.083698 19416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:51.083715 19416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:51.089311 19416 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:34395
I20260806 13:06:51.089375 19594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:34395 every 8 connection(s)
I20260806 13:06:51.089660 19416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:06:51.093029 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 19416
I20260806 13:06:51.093116 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:06:51.093276 14209 external_mini_cluster.cc:1428] Setting key 9123d09b8ec275bc4bead607ebbc75d4
Aug 06 13:06:51 dist-test-slave-q9h9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021611, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:51.098857 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:51.090345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:34303 (local address 127.13.224.126:43235)
0806 13:06:51.090407 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:51.090408 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:51.090600 (+ 192us) server_negotiation.cc:407] Connection header received
0806 13:06:51.091511 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:51.091513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:51.091527 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:51.091577 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:51.092174 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.092686 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.093773 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.094046 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.094107 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:51.096142 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:51.096143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:51.096144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:51.096152 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:51.097551 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:51.097934 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:51.097936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:51.097936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:51.098002 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:51.098243 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:51.098245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:51.098245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:51.098365 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:51.098423 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:51.098666 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 13:06:51.098763 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 13:06:51.098989 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:51.090298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:43235 (local address 127.13.224.65:34303)
0806 13:06:51.090484 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:51.090525 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:51.091395 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:51.091663 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:51.091667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:51.091909 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:51.092083 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:51.092087 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.092768 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.092769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:51.093699 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:51.093702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.094117 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.094118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:51.094238 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:51.094655 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:51.094663 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:51.096054 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:51.097647 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:51.097653 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:51.097655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:51.097860 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:51.098083 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:51.098084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:51.098084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:51.098178 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:51.098421 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:51.098428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:51.098544 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 13:06:51.098644 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 13:06:51.098787 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
I20260806 13:06:51.099654 19595 heartbeater.cc:347] Connected to a master server at 127.13.224.126:43235
I20260806 13:06:51.099731 19595 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:51.099884 19595 heartbeater.cc:511] Master 127.13.224.126:43235 requested a full tablet report, sending...
I20260806 13:06:51.100386 19327 ts_manager.cc:194] Registered new tserver with Master: 8ee0575513254f53af53d66624949076 (127.13.224.65:34395)
I20260806 13:06:51.100966 19327 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:34303
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:06:51.116425 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43235
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:51.195570 19602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:51.195755 19602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:51.195776 19602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:51.196913 19602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:51.196954 19602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:51.196979 19602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:06:51.198519 19602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:51.198729 19602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:51.198931 19602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:51.203570 19612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:51.203583 19609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:51.203569 19610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:51.203970 19602 server_base.cc:1034] running on GCE node
I20260806 13:06:51.204142 19602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:51.204344 19602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:51.205482 19602 hybrid_clock.cc:648] HybridClock initialized: now 1786021611205467 us; error 28 us; skew 500 ppm
Aug 06 13:06:51 dist-test-slave-q9h9 krb5kdc[19265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021611, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:51.207788 19602 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:06:51.208166 19602 webserver.cc:460] Webserver started at http://127.13.224.66:35571/ using document root <none> and password file <none>
I20260806 13:06:51.208356 19602 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:51.208398 19602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:51.208513 19602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:51.209451 19602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "75adf79480f543848cd20a1fca52260d"
format_stamp: "Formatted at 2026-08-06 13:06:51 on dist-test-slave-q9h9"
server_key: "01a5e0179691af9fa454f75f7b6ba46b"
server_key_iv: "629da6b678a17546a794b6edd91aea39"
server_key_version: "encryptionkey@0"
I20260806 13:06:51.209764 19602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "75adf79480f543848cd20a1fca52260d"
format_stamp: "Formatted at 2026-08-06 13:06:51 on dist-test-slave-q9h9"
server_key: "01a5e0179691af9fa454f75f7b6ba46b"
server_key_iv: "629da6b678a17546a794b6edd91aea39"
server_key_version: "encryptionkey@0"
I20260806 13:06:51.211045 19602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:06:51.211668 19620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:51.211824 19602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:51.211896 19602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "75adf79480f543848cd20a1fca52260d"
format_stamp: "Formatted at 2026-08-06 13:06:51 on dist-test-slave-q9h9"
server_key: "01a5e0179691af9fa454f75f7b6ba46b"
server_key_iv: "629da6b678a17546a794b6edd91aea39"
server_key_version: "encryptionkey@0"
I20260806 13:06:51.211961 19602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:51.225548 19602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:51.227082 19602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:51.227198 19602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:51.227380 19602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:51.227641 19602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:51.227674 19602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:51.227700 19602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:51.227725 19602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:51.234215 19602 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:42573
I20260806 13:06:51.234287 19789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:42573 every 8 connection(s)
I20260806 13:06:51.234552 19602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
Aug 06 13:06:51 dist-test-slave-q9h9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021611, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:51.240661 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 19602
I20260806 13:06:51.240782 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:06:51.240950 14209 external_mini_cluster.cc:1428] Setting key 2b8fca3dbcbb85b58e7edd7551418e41
I20260806 13:06:51.242772 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:51.235167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:40535 (local address 127.13.224.126:43235)
0806 13:06:51.235206 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:51.235208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:51.235402 (+ 194us) server_negotiation.cc:407] Connection header received
0806 13:06:51.236197 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:51.236198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:51.236207 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:51.236264 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:51.236787 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.237277 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.238209 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.238407 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.238454 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:51.240242 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:51.240244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:51.240244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:51.240251 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:51.241584 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:51.241933 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:51.241935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:51.241935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:51.241995 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:51.242229 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:51.242230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:51.242230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:51.242363 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:51.242425 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:51.242616 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 13:06:51.242690 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:06:51.242848 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:51.235129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:43235 (local address 127.13.224.66:40535)
0806 13:06:51.235294 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:51.235331 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:51.236120 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:51.236314 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:51.236317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:51.236539 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:51.236724 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:51.236727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.237336 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.237337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:51.238143 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:51.238145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.238463 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.238464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:51.238560 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:51.238936 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:51.238943 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:51.240156 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:51.241665 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:51.241672 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:51.241674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:51.241873 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:51.242089 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:51.242090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:51.242090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:51.242177 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:51.242417 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:51.242419 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:51.242515 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 13:06:51.242586 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 13:06:51.242691 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260806 13:06:51.243266 19790 heartbeater.cc:347] Connected to a master server at 127.13.224.126:43235
I20260806 13:06:51.243362 19790 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:51.243534 19790 heartbeater.cc:511] Master 127.13.224.126:43235 requested a full tablet report, sending...
I20260806 13:06:51.243857 19327 ts_manager.cc:194] Registered new tserver with Master: 75adf79480f543848cd20a1fca52260d (127.13.224.66:42573)
I20260806 13:06:51.244208 19327 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:40535
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:06:51.263796 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43235
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:51.342519 19798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:51.342693 19798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:51.342722 19798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:51.343819 19798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:51.343855 19798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:51.343887 19798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:06:51.345347 19798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:51.345572 19798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:51.345769 19798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:51.350270 19807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:51.350312 19813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:51.350334 19808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:51.350560 19798 server_base.cc:1034] running on GCE node
I20260806 13:06:51.350757 19798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:51.350941 19798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:51.352065 19798 hybrid_clock.cc:648] HybridClock initialized: now 1786021611352054 us; error 28 us; skew 500 ppm
Aug 06 13:06:51 dist-test-slave-q9h9 krb5kdc[19265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021611, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:51.354272 19798 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:06:51.354655 19798 webserver.cc:460] Webserver started at http://127.13.224.67:36933/ using document root <none> and password file <none>
I20260806 13:06:51.354871 19798 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:51.354918 19798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:51.355037 19798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:51.355954 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "6841fbab4c1443d5b6fb3a481e178849"
format_stamp: "Formatted at 2026-08-06 13:06:51 on dist-test-slave-q9h9"
server_key: "6ca711ce77ecd6e340cc2ea82aa1a1ad"
server_key_iv: "c9ce39ddbae41e35f785c7671ecc188e"
server_key_version: "encryptionkey@0"
I20260806 13:06:51.356271 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "6841fbab4c1443d5b6fb3a481e178849"
format_stamp: "Formatted at 2026-08-06 13:06:51 on dist-test-slave-q9h9"
server_key: "6ca711ce77ecd6e340cc2ea82aa1a1ad"
server_key_iv: "c9ce39ddbae41e35f785c7671ecc188e"
server_key_version: "encryptionkey@0"
I20260806 13:06:51.357409 19798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:06:51.358035 19825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:51.358173 19798 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:51.358261 19798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "6841fbab4c1443d5b6fb3a481e178849"
format_stamp: "Formatted at 2026-08-06 13:06:51 on dist-test-slave-q9h9"
server_key: "6ca711ce77ecd6e340cc2ea82aa1a1ad"
server_key_iv: "c9ce39ddbae41e35f785c7671ecc188e"
server_key_version: "encryptionkey@0"
I20260806 13:06:51.358323 19798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:51.378425 19798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:51.380023 19798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:51.380143 19798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:51.380327 19798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:51.380601 19798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:51.380635 19798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:51.380661 19798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:51.380686 19798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:51.386085 19798 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:35789
I20260806 13:06:51.386142 20005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:35789 every 8 connection(s)
I20260806 13:06:51.386415 19798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:06:51.387799 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 19798
I20260806 13:06:51.387892 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:06:51.388060 14209 external_mini_cluster.cc:1428] Setting key 468d3be45dc6fcc96ae60482008b8b87
Aug 06 13:06:51 dist-test-slave-q9h9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021611, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:51.394963 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:51.387104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:54383 (local address 127.13.224.126:43235)
0806 13:06:51.387152 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:51.387154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:51.387312 (+ 158us) server_negotiation.cc:407] Connection header received
0806 13:06:51.388222 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:51.388223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:51.388250 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:51.388301 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:51.389008 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.389510 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.390530 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.390746 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.390799 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:51.392691 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:51.392693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:51.392693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:51.392700 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:51.393711 (+ 1011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:51.394088 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:51.394090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:51.394090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:51.394146 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:51.394362 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:51.394363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:51.394363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:51.394473 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:51.394528 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:51.394782 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 13:06:51.394872 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:06:51.395112 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:51.387067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:43235 (local address 127.13.224.67:54383)
0806 13:06:51.387202 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:51.387237 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:51.388121 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:51.388388 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:51.388391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:51.388691 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:51.388935 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:51.388940 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.389591 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.389592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:51.390455 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:51.390457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.390816 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.390816 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:51.390958 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:51.391350 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:51.391358 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:51.392609 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:51.393784 (+ 1175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:51.393791 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:51.393793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:51.393993 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:51.394199 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:51.394200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:51.394201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:51.394305 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:51.394529 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:51.394536 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:51.394662 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 13:06:51.394754 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 13:06:51.394912 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260806 13:06:51.395530 20009 heartbeater.cc:347] Connected to a master server at 127.13.224.126:43235
I20260806 13:06:51.395615 20009 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:51.395778 20009 heartbeater.cc:511] Master 127.13.224.126:43235 requested a full tablet report, sending...
I20260806 13:06:51.396107 19327 ts_manager.cc:194] Registered new tserver with Master: 6841fbab4c1443d5b6fb3a481e178849 (127.13.224.67:35789)
I20260806 13:06:51.396446 19327 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:54383
I20260806 13:06:51.399506 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 13:06:51 dist-test-slave-q9h9 krb5kdc[19265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 13:06:51 dist-test-slave-q9h9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.65@KRBTEST.COM
I20260806 13:06:51.421226 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:51.414594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55006 (local address 127.13.224.65:34395)
0806 13:06:51.414761 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:51.414766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:51.414778 (+ 12us) server_negotiation.cc:407] Connection header received
0806 13:06:51.414813 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:51.414816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:51.414831 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:51.414894 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:51.415396 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.416140 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.416821 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.417102 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.417149 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:51.419098 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:51.419106 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:51.419106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:51.419113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:51.420370 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:51.420652 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:51.420653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:51.420654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:51.420701 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:51.420827 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:51.420827 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:51.420827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:51.420933 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:51.421019 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:51.421077 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 13:06:51.421134 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
Aug 06 13:06:51 dist-test-slave-q9h9 krb5kdc[19265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:51.428006 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:51.422366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59944 (local address 127.13.224.126:43235)
0806 13:06:51.422405 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:51.422406 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:51.422440 (+ 34us) server_negotiation.cc:407] Connection header received
0806 13:06:51.422488 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:51.422489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:51.422496 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:51.422537 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:51.422932 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.423377 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.423939 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:51.424102 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:51.424141 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:51.425883 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:51.425884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:51.425885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:51.425891 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:51.426942 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:51.427324 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:51.427325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:51.427325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:51.427401 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:51.427614 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:51.427614 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:51.427615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:51.427739 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:51.427811 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:51.427875 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 13:06:51.427936 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 13:06:51.428232 19327 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59944
I20260806 13:06:51.428526 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 19416
I20260806 13:06:51.444511 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 19602
I20260806 13:06:51.452517 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 19798
I20260806 13:06:51.462842 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 19284
2026-08-06T13:06:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1879 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:06:51 dist-test-slave-q9h9 krb5kdc[20025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:06:51 dist-test-slave-q9h9 krb5kdc[20025](info): set up 2 sockets
Aug 06 13:06:51 dist-test-slave-q9h9 krb5kdc[20025](info): commencing operation
krb5kdc: starting...
W20260806 13:06:52.570907 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:06:52 dist-test-slave-q9h9 krb5kdc[20025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:06:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:06:52.678586 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:34221
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:35369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:34221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:52.757089 20043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:52.757279 20043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:52.757300 20043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:52.758451 20043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:52.758490 20043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:52.758503 20043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:52.758527 20043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:52.758543 20043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:06:52.760020 20043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:35369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:34221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:34221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:52.760229 20043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:52.760429 20043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 13:06:52.765029 20043 server_base.cc:1034] running on GCE node
W20260806 13:06:52.764923 20055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:52.764933 20052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:52.764926 20053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:52.765388 20043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:52.765653 20043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:52.766788 20043 hybrid_clock.cc:648] HybridClock initialized: now 1786021612766778 us; error 33 us; skew 500 ppm
Aug 06 13:06:52 dist-test-slave-q9h9 krb5kdc[20025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021612, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:52.769032 20043 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:06:52.769373 20043 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:52.769426 20043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:52.769550 20043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:52.770546 20043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "713afb16224b4bb4879d1860158a8126"
format_stamp: "Formatted at 2026-08-06 13:06:52 on dist-test-slave-q9h9"
server_key: "461b6039354a1988479e570f84309259"
server_key_iv: "089d2070fc98a9021a5dfaf2ff3ab0f1"
server_key_version: "encryptionkey@0"
I20260806 13:06:52.770864 20043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "713afb16224b4bb4879d1860158a8126"
format_stamp: "Formatted at 2026-08-06 13:06:52 on dist-test-slave-q9h9"
server_key: "461b6039354a1988479e570f84309259"
server_key_iv: "089d2070fc98a9021a5dfaf2ff3ab0f1"
server_key_version: "encryptionkey@0"
I20260806 13:06:52.772014 20043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:06:52.772651 20063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:52.772825 20043 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:52.772907 20043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "713afb16224b4bb4879d1860158a8126"
format_stamp: "Formatted at 2026-08-06 13:06:52 on dist-test-slave-q9h9"
server_key: "461b6039354a1988479e570f84309259"
server_key_iv: "089d2070fc98a9021a5dfaf2ff3ab0f1"
server_key_version: "encryptionkey@0"
I20260806 13:06:52.772967 20043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:52.786327 20043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:52.787870 20043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:52.787997 20043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:52.791649 20043 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:34221
I20260806 13:06:52.791675 20137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:34221 every 8 connection(s)
I20260806 13:06:52.791916 20043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:52.792412 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 20043
I20260806 13:06:52.792498 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:52.792663 14209 external_mini_cluster.cc:1428] Setting key 6c314a131f6033a26db47d25ae1ab873
I20260806 13:06:52.792961 20138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:52.796101 20138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126: Bootstrap starting.
I20260806 13:06:52.796715 20138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:52.797006 20138 log.cc:824] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:52.797706 20138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126: No bootstrap required, opened a new log
Aug 06 13:06:52 dist-test-slave-q9h9 krb5kdc[20025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:52.799683 20138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713afb16224b4bb4879d1860158a8126" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34221 } }
I20260806 13:06:52.799871 20138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:52.799924 20138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 713afb16224b4bb4879d1860158a8126, State: Initialized, Role: FOLLOWER
I20260806 13:06:52.800050 20138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713afb16224b4bb4879d1860158a8126" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34221 } }
I20260806 13:06:52.800113 20138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:52.800143 20138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:52.800189 20138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:52.800806 20138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713afb16224b4bb4879d1860158a8126" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34221 } }
I20260806 13:06:52.800916 20138 leader_election.cc:302] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 713afb16224b4bb4879d1860158a8126; no voters:
I20260806 13:06:52.801072 20138 leader_election.cc:288] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:52.801126 20145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:52.801291 20145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [term 1 LEADER]: Becoming Leader. State: Replica: 713afb16224b4bb4879d1860158a8126, State: Running, Role: LEADER
I20260806 13:06:52.801322 20138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:52.801380 20145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713afb16224b4bb4879d1860158a8126" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34221 } }
I20260806 13:06:52.801702 20145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 713afb16224b4bb4879d1860158a8126. Latest consensus state: current_term: 1 leader_uuid: "713afb16224b4bb4879d1860158a8126" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713afb16224b4bb4879d1860158a8126" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34221 } } }
I20260806 13:06:52.802141 20148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "713afb16224b4bb4879d1860158a8126" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "713afb16224b4bb4879d1860158a8126" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34221 } } }
I20260806 13:06:52.802210 20145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:52.802212 20148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:52.802628 20159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:52.803134 20159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:52.803980 20143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:52.793134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.13.224.126:34221)
0806 13:06:52.793318 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:52.793328 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:52.793360 (+ 32us) server_negotiation.cc:407] Connection header received
0806 13:06:52.794076 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:52.794085 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:52.794149 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:52.794308 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:52.794886 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:52.796175 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:52.796929 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:52.797451 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:52.797559 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:52.799991 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:52.799995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:52.799996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:52.800007 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:52.802310 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:52.802724 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:52.802726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:52.802727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:52.802835 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:52.803056 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:52.803057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:52.803058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:52.803343 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:52.803493 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:52.803654 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 13:06:52.803797 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260806 13:06:52.805039 20159 catalog_manager.cc:1347] Generated new cluster ID: 734a2956c8254ae195e08ef337b2bcd9
I20260806 13:06:52.805080 20159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:06:52.809110 20159 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:52.809798 20159 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:52.813730 20159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126: Generated new TSK 0
I20260806 13:06:52.813891 20159 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:06:52.837764 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:34221
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:35369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:52.918423 20177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:52.918601 20177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:52.918624 20177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:52.919739 20177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:52.919782 20177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:52.919808 20177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:06:52.921283 20177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:35369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:34221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:52.921489 20177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:52.921693 20177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:52.926335 20184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:52.926344 20185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:52.926400 20189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:52.926374 20177 server_base.cc:1034] running on GCE node
I20260806 13:06:52.926664 20177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:52.926867 20177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:52.928022 20177 hybrid_clock.cc:648] HybridClock initialized: now 1786021612928010 us; error 36 us; skew 500 ppm
Aug 06 13:06:52 dist-test-slave-q9h9 krb5kdc[20025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021612, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:52.930307 20177 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:06:52.930619 20177 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:52.930668 20177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:52.930796 20177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:52.931815 20177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "3b3c5f5635c146a5bc04c6e5a0700259"
format_stamp: "Formatted at 2026-08-06 13:06:52 on dist-test-slave-q9h9"
server_key: "32ff3cc2f9776c60c49c37b6966afafc"
server_key_iv: "f96878e89d8022476c00216aa7e7c77f"
server_key_version: "encryptionkey@0"
I20260806 13:06:52.932148 20177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "3b3c5f5635c146a5bc04c6e5a0700259"
format_stamp: "Formatted at 2026-08-06 13:06:52 on dist-test-slave-q9h9"
server_key: "32ff3cc2f9776c60c49c37b6966afafc"
server_key_iv: "f96878e89d8022476c00216aa7e7c77f"
server_key_version: "encryptionkey@0"
I20260806 13:06:52.933279 20177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 13:06:52.933904 20197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:52.934079 20177 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:52.934161 20177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "3b3c5f5635c146a5bc04c6e5a0700259"
format_stamp: "Formatted at 2026-08-06 13:06:52 on dist-test-slave-q9h9"
server_key: "32ff3cc2f9776c60c49c37b6966afafc"
server_key_iv: "f96878e89d8022476c00216aa7e7c77f"
server_key_version: "encryptionkey@0"
I20260806 13:06:52.934219 20177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:52.941347 20177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:52.942993 20177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:52.943120 20177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:52.943302 20177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:52.943567 20177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:52.943600 20177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:52.943625 20177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:52.943650 20177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:52.948972 20177 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:45025
I20260806 13:06:52.949028 20359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:45025 every 8 connection(s)
I20260806 13:06:52.949225 20177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:06:52.952329 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 20177
I20260806 13:06:52.952428 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:06:52.952608 14209 external_mini_cluster.cc:1428] Setting key 18d516e8d35d464aeeb61d9cbc40d0d6
Aug 06 13:06:52 dist-test-slave-q9h9 krb5kdc[20025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021612, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:52.958475 20143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:52.949881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:48679 (local address 127.13.224.126:34221)
0806 13:06:52.949921 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:52.949923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:52.950124 (+ 201us) server_negotiation.cc:407] Connection header received
0806 13:06:52.950534 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:52.950535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:52.950543 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:52.950592 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:52.951405 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:52.951888 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:52.953031 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:52.953311 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:52.953365 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:52.955704 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:52.955705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:52.955706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:52.955712 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:52.957095 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:52.957465 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:52.957467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:52.957467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:52.957540 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:52.957822 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:52.957823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:52.957824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:52.957965 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:52.958051 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:52.958296 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 13:06:52.958385 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:06:52.958509 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:52.949850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:34221 (local address 127.13.224.65:48679)
0806 13:06:52.950007 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:52.950065 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:52.950460 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:52.950644 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:52.950647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:52.950879 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:52.951335 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:52.951339 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:52.951944 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:52.951944 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:52.952961 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:52.952964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:52.953393 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:52.953394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:52.953530 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:52.953951 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:52.953960 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:52.955615 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:52.957179 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:52.957185 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:52.957187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:52.957387 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:52.957607 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:52.957607 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:52.957607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:52.957692 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:52.958050 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:52.958054 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:52.958174 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 13:06:52.958254 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 13:06:52.958364 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260806 13:06:52.959003 20360 heartbeater.cc:347] Connected to a master server at 127.13.224.126:34221
I20260806 13:06:52.959092 20360 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:52.959227 20360 heartbeater.cc:511] Master 127.13.224.126:34221 requested a full tablet report, sending...
I20260806 13:06:52.959759 20092 ts_manager.cc:194] Registered new tserver with Master: 3b3c5f5635c146a5bc04c6e5a0700259 (127.13.224.65:45025)
I20260806 13:06:52.960630 20092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:48679
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:06:52.976722 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:34221
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:35369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:53.057880 20367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:53.058099 20367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:53.058127 20367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:53.059228 20367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:53.059262 20367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:53.059286 20367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:06:53.060782 20367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:35369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:34221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:53.060961 20367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:53.061137 20367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:53.065750 20376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:53.065752 20373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:53.065876 20367 server_base.cc:1034] running on GCE node
W20260806 13:06:53.065749 20374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:53.066187 20367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:53.066416 20367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:53.067551 20367 hybrid_clock.cc:648] HybridClock initialized: now 1786021613067534 us; error 32 us; skew 500 ppm
Aug 06 13:06:53 dist-test-slave-q9h9 krb5kdc[20025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:53.069839 20367 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:06:53.070195 20367 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:53.070250 20367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:53.070371 20367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:53.071364 20367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "d4b928cef62841658fa4881246d48c1e"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "4229603dda7aa73b10cd24d6841448c1"
server_key_iv: "a5ed65854141e1701f15c58461d28b50"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.071677 20367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "d4b928cef62841658fa4881246d48c1e"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "4229603dda7aa73b10cd24d6841448c1"
server_key_iv: "a5ed65854141e1701f15c58461d28b50"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.072822 20367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:06:53.073447 20385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.073599 20367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:53.073681 20367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "d4b928cef62841658fa4881246d48c1e"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "4229603dda7aa73b10cd24d6841448c1"
server_key_iv: "a5ed65854141e1701f15c58461d28b50"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.073745 20367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:53.085731 20367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:53.087410 20367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:53.087543 20367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:53.087730 20367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:53.088025 20367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:53.088065 20367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.088092 20367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:53.088117 20367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.093667 20367 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:37165
I20260806 13:06:53.093699 20554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:37165 every 8 connection(s)
I20260806 13:06:53.093940 20367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
Aug 06 13:06:53 dist-test-slave-q9h9 krb5kdc[20025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:53.101303 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 20367
I20260806 13:06:53.101408 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:06:53.101603 14209 external_mini_cluster.cc:1428] Setting key 68034a17f0508d113ae70efcae3e62eb
I20260806 13:06:53.102207 20143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.094609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:48659 (local address 127.13.224.126:34221)
0806 13:06:53.094664 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.094666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.094864 (+ 198us) server_negotiation.cc:407] Connection header received
0806 13:06:53.095299 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.095300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.095309 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.095361 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:53.096182 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.096668 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.097587 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.097761 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.097803 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.099600 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:53.099602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:53.099602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:53.099609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:53.100856 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:53.101226 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:53.101227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:53.101228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:53.101295 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:53.101534 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:53.101535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:53.101536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:53.101672 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:53.101757 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.101999 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.102098 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 13:06:53.102362 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.094574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:34221 (local address 127.13.224.66:48659)
0806 13:06:53.094734 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.094775 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:53.095219 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:53.095411 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:53.095414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:53.095646 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:53.096115 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.096120 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.096739 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.096740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.097525 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.097527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.097812 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.097813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.097903 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.098283 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:53.098290 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:53.099521 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:53.100917 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:53.100920 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:53.100922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:53.101163 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:53.101362 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:53.101362 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:53.101363 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:53.101486 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:53.101739 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:53.101745 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:53.101893 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 13:06:53.101976 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 13:06:53.102132 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260806 13:06:53.102766 20555 heartbeater.cc:347] Connected to a master server at 127.13.224.126:34221
I20260806 13:06:53.102843 20555 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:53.102975 20555 heartbeater.cc:511] Master 127.13.224.126:34221 requested a full tablet report, sending...
I20260806 13:06:53.103292 20092 ts_manager.cc:194] Registered new tserver with Master: d4b928cef62841658fa4881246d48c1e (127.13.224.66:37165)
I20260806 13:06:53.103662 20092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:48659
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:06:53.122823 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:34221
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:35369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:53.201555 20562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:53.201737 20562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:53.201761 20562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:53.202948 20562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:53.202989 20562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:53.203015 20562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:06:53.204496 20562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:35369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:34221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:53.204671 20562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:53.204838 20562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:53.209419 20570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:53.209421 20572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:53.209434 20577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:53.209448 20562 server_base.cc:1034] running on GCE node
I20260806 13:06:53.209853 20562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:53.210111 20562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:53.211256 20562 hybrid_clock.cc:648] HybridClock initialized: now 1786021613211244 us; error 27 us; skew 500 ppm
Aug 06 13:06:53 dist-test-slave-q9h9 krb5kdc[20025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:53.213492 20562 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:06:53.213791 20562 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:53.213841 20562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:53.213963 20562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:53.214917 20562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "5cb4844b20d9de4b2dac80ec9af18411"
server_key_iv: "90189dfb9fba882c9635688de0ed578e"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.215222 20562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "5cb4844b20d9de4b2dac80ec9af18411"
server_key_iv: "90189dfb9fba882c9635688de0ed578e"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.216355 20562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:06:53.216984 20586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.217123 20562 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:53.217204 20562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "5cb4844b20d9de4b2dac80ec9af18411"
server_key_iv: "90189dfb9fba882c9635688de0ed578e"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.217262 20562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:53.228173 20562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:53.229673 20562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:53.229813 20562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:53.230005 20562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:53.230298 20562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:53.230331 20562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.230357 20562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:53.230374 20562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.235667 20562 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:42689
I20260806 13:06:53.235728 20757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:42689 every 8 connection(s)
I20260806 13:06:53.235944 20562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:06:53.236688 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 20562
I20260806 13:06:53.236835 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:06:53.237112 14209 external_mini_cluster.cc:1428] Setting key 769eae610af3f4610786aac6b0dbae3b
Aug 06 13:06:53 dist-test-slave-q9h9 krb5kdc[20025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021613, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:53.245306 20143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.236516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:43645 (local address 127.13.224.126:34221)
0806 13:06:53.236586 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.236588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.236773 (+ 185us) server_negotiation.cc:407] Connection header received
0806 13:06:53.238153 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.238155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.238167 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.238226 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:53.239342 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.239963 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.240908 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.241119 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.241177 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.243196 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:53.243197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:53.243197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:53.243205 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:53.244273 (+ 1068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:53.244627 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:53.244628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:53.244628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:53.244671 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:53.244837 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:53.244837 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:53.244838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:53.244930 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:53.244974 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.245190 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.245244 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 13:06:53.245512 20760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.236500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:34221 (local address 127.13.224.67:43645)
0806 13:06:53.236661 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.236698 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:53.237296 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:53.238300 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:53.238303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:53.238598 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:53.239204 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.239211 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.240056 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.240057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.240837 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.240839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.241222 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.241224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.241338 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.241733 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:53.241741 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:53.243113 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:53.244344 (+ 1231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:53.244351 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:53.244353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:53.244567 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:53.244707 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:53.244707 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:53.244707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:53.244794 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:53.244974 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:53.244977 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:53.245094 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 13:06:53.245175 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 13:06:53.245291 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260806 13:06:53.245921 20758 heartbeater.cc:347] Connected to a master server at 127.13.224.126:34221
I20260806 13:06:53.246006 20758 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:53.246178 20758 heartbeater.cc:511] Master 127.13.224.126:34221 requested a full tablet report, sending...
I20260806 13:06:53.246537 20092 ts_manager.cc:194] Registered new tserver with Master: 3350e4148f2d4ffbb75f6b9cb2268bb5 (127.13.224.67:42689)
I20260806 13:06:53.246883 20092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:43645
I20260806 13:06:53.248790 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:06:53.254889 20143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.249947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41946 (local address 127.13.224.126:34221)
0806 13:06:53.250008 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.250010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.250108 (+ 98us) server_negotiation.cc:407] Connection header received
0806 13:06:53.250157 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.250157 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.250165 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.250208 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:53.250699 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.251137 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.251763 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.251924 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.251963 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.252885 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:53.252886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:53.252886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:53.252891 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:53.254044 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:53.254326 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:53.254327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:53.254327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:53.254372 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:53.254493 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:53.254493 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:53.254494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:53.254613 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:53.254684 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.254750 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.254822 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 13:06:53.256145 20092 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:06:53.257566 20092 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:06:53.263303 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.259657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:45025 (local address 127.0.0.1:37218)
0806 13:06:53.259867 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.259876 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:53.259919 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:53.260165 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:53.260168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:53.260170 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:53.260231 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.260232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.261120 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.261122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.262196 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.262199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.263021 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.263022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.263097 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.263110 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 13:06:53.263172 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 13:06:53.263223 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":63,"threads_started":1}
I20260806 13:06:53.263316 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.259866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37218 (local address 127.13.224.65:45025)
0806 13:06:53.260016 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.260020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.260028 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:06:53.260060 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.260062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.260073 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.260132 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:53.260297 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.261027 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.262290 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.262949 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.263008 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.263062 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.263196 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.263234 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260806 13:06:53.263757 20773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.259454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:37165 (local address 127.0.0.1:35806)
0806 13:06:53.259789 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.259797 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:53.259840 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:53.260454 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:53.260454 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:53.260456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:53.260486 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.260488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.261732 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.261733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.262694 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.262697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.263530 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.263531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.263595 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.263599 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:06:53.263656 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 13:06:53.263692 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":77,"threads_started":1}
I20260806 13:06:53.263797 20769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.259543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35806 (local address 127.13.224.66:37165)
0806 13:06:53.260038 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.260042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.260052 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:06:53.260087 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.260088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.260100 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.260559 (+ 459us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:53.260647 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.261652 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.262789 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.263455 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.263522 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.263578 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.263666 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.263707 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":89,"threads_started":1}
I20260806 13:06:53.263974 20767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.259453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:42689 (local address 127.0.0.1:53546)
0806 13:06:53.259620 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.259654 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:53.259711 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:53.260356 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:53.260357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:53.260358 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:53.260391 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.260392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.261600 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.261601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.262676 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.262678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.263698 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.263699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.263758 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.263761 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:06:53.263869 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 13:06:53.263911 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":23,"thread_start_us":73,"threads_started":1}
I20260806 13:06:53.264173 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.259549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53546 (local address 127.13.224.67:42689)
0806 13:06:53.260226 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.260230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.260238 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:06:53.260278 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.260280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.260294 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.260358 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:53.260461 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.261496 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.262757 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.263566 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.263620 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.263665 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.264062 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.264101 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":87,"threads_started":1}
I20260806 13:06:53.264881 20264 tablet_service.cc:1467] Processing CreateTablet for tablet 344696fbe916417ba36b4640f9faa501 (DEFAULT_TABLE table=test-table [id=e49fa65ddf1147d4a5bef385bc2cd50e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:53.264914 20657 tablet_service.cc:1467] Processing CreateTablet for tablet 344696fbe916417ba36b4640f9faa501 (DEFAULT_TABLE table=test-table [id=e49fa65ddf1147d4a5bef385bc2cd50e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:53.265239 20264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 344696fbe916417ba36b4640f9faa501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:53.265239 20657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 344696fbe916417ba36b4640f9faa501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:53.265514 20465 tablet_service.cc:1467] Processing CreateTablet for tablet 344696fbe916417ba36b4640f9faa501 (DEFAULT_TABLE table=test-table [id=e49fa65ddf1147d4a5bef385bc2cd50e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:53.265834 20465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 344696fbe916417ba36b4640f9faa501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:53.267813 20776 tablet_bootstrap.cc:492] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259: Bootstrap starting.
I20260806 13:06:53.268488 20776 tablet_bootstrap.cc:654] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:53.268462 20778 tablet_bootstrap.cc:492] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5: Bootstrap starting.
I20260806 13:06:53.268913 20779 tablet_bootstrap.cc:492] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e: Bootstrap starting.
I20260806 13:06:53.269078 20776 log.cc:824] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:53.269114 20778 tablet_bootstrap.cc:654] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:53.269609 20778 log.cc:824] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:53.269651 20779 tablet_bootstrap.cc:654] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:53.269869 20776 tablet_bootstrap.cc:492] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259: No bootstrap required, opened a new log
I20260806 13:06:53.269932 20776 ts_tablet_manager.cc:1397] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:06:53.270233 20779 log.cc:824] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:53.270515 20778 tablet_bootstrap.cc:492] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5: No bootstrap required, opened a new log
I20260806 13:06:53.270587 20778 ts_tablet_manager.cc:1397] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:06:53.270869 20779 tablet_bootstrap.cc:492] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e: No bootstrap required, opened a new log
I20260806 13:06:53.270928 20779 ts_tablet_manager.cc:1397] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:06:53.271788 20776 raft_consensus.cc:348] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b928cef62841658fa4881246d48c1e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37165 } } peers { permanent_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 45025 } } peers { permanent_uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42689 } }
I20260806 13:06:53.271953 20776 raft_consensus.cc:374] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:53.271988 20776 raft_consensus.cc:729] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b3c5f5635c146a5bc04c6e5a0700259, State: Initialized, Role: FOLLOWER
I20260806 13:06:53.272104 20776 consensus_queue.cc:260] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b928cef62841658fa4881246d48c1e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37165 } } peers { permanent_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 45025 } } peers { permanent_uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42689 } }
I20260806 13:06:53.272389 20776 ts_tablet_manager.cc:1428] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 13:06:53.272301 20779 raft_consensus.cc:348] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b928cef62841658fa4881246d48c1e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37165 } } peers { permanent_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 45025 } } peers { permanent_uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42689 } }
I20260806 13:06:53.272445 20779 raft_consensus.cc:374] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:53.272490 20779 raft_consensus.cc:729] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4b928cef62841658fa4881246d48c1e, State: Initialized, Role: FOLLOWER
I20260806 13:06:53.272503 20360 heartbeater.cc:503] Master 127.13.224.126:34221 was elected leader, sending a full tablet report...
I20260806 13:06:53.272466 20778 raft_consensus.cc:348] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b928cef62841658fa4881246d48c1e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37165 } } peers { permanent_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 45025 } } peers { permanent_uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42689 } }
I20260806 13:06:53.272599 20778 raft_consensus.cc:374] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:53.272590 20779 consensus_queue.cc:260] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b928cef62841658fa4881246d48c1e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37165 } } peers { permanent_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 45025 } } peers { permanent_uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42689 } }
I20260806 13:06:53.272632 20778 raft_consensus.cc:729] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3350e4148f2d4ffbb75f6b9cb2268bb5, State: Initialized, Role: FOLLOWER
I20260806 13:06:53.272785 20778 consensus_queue.cc:260] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b928cef62841658fa4881246d48c1e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37165 } } peers { permanent_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 45025 } } peers { permanent_uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42689 } }
I20260806 13:06:53.272864 20779 ts_tablet_manager.cc:1428] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:06:53.272969 20555 heartbeater.cc:503] Master 127.13.224.126:34221 was elected leader, sending a full tablet report...
I20260806 13:06:53.273058 20778 ts_tablet_manager.cc:1428] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 13:06:53.273182 20758 heartbeater.cc:503] Master 127.13.224.126:34221 was elected leader, sending a full tablet report...
W20260806 13:06:53.344453 20557 tablet.cc:2369] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:06:53.424690 20787 raft_consensus.cc:482] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:06:53.424817 20787 raft_consensus.cc:504] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b928cef62841658fa4881246d48c1e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37165 } } peers { permanent_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 45025 } } peers { permanent_uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42689 } }
I20260806 13:06:53.425137 20787 leader_election.cc:288] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4b928cef62841658fa4881246d48c1e (127.13.224.66:37165), 3350e4148f2d4ffbb75f6b9cb2268bb5 (127.13.224.67:42689)
I20260806 13:06:53.427946 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.425381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:54903 (local address 127.13.224.67:42689)
0806 13:06:53.425441 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.425443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.425476 (+ 33us) server_negotiation.cc:407] Connection header received
0806 13:06:53.425500 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.425501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.425509 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.425555 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:53.425694 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.426225 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.427252 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.427627 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.427690 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.427700 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.427846 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.427882 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 13:06:53.427946 20363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.425304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:42689 (local address 127.13.224.65:54903)
0806 13:06:53.425368 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.425379 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:53.425423 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:53.425573 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:53.425575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:53.425576 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:53.425632 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.425634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.426308 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.426309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.427148 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.427151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.427708 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.427709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.427780 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.427783 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:06:53.427826 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 13:06:53.427863 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":53}
I20260806 13:06:53.428035 20769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.425387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:42349 (local address 127.13.224.66:37165)
0806 13:06:53.425440 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.425443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.425635 (+ 192us) server_negotiation.cc:407] Connection header received
0806 13:06:53.425671 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.425672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.425681 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.425729 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:06:53.425883 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.426366 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.427317 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.427701 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.427764 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.427774 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.427923 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.427964 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 13:06:53.428064 20790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.425306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:37165 (local address 127.13.224.65:42349)
0806 13:06:53.425567 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.425576 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:53.425624 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:53.425731 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:53.425732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:53.425733 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:06:53.425817 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.425819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.426443 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.426444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.427224 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:53.427227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.427784 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.427785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:53.427861 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.427865 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:06:53.427909 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 13:06:53.427946 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":106,"thread_start_us":98,"threads_started":1}
I20260806 13:06:53.428354 20692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344696fbe916417ba36b4640f9faa501" candidate_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5" is_pre_election: true
I20260806 13:06:53.428392 20493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344696fbe916417ba36b4640f9faa501" candidate_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4b928cef62841658fa4881246d48c1e" is_pre_election: true
I20260806 13:06:53.428524 20493 raft_consensus.cc:2454] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b3c5f5635c146a5bc04c6e5a0700259 in term 0.
I20260806 13:06:53.428525 20692 raft_consensus.cc:2454] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b3c5f5635c146a5bc04c6e5a0700259 in term 0.
I20260806 13:06:53.428761 20198 leader_election.cc:302] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b3c5f5635c146a5bc04c6e5a0700259, d4b928cef62841658fa4881246d48c1e; no voters:
I20260806 13:06:53.428901 20787 raft_consensus.cc:2781] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:06:53.428961 20787 raft_consensus.cc:482] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:06:53.428989 20787 raft_consensus.cc:3037] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:53.429630 20787 raft_consensus.cc:504] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b928cef62841658fa4881246d48c1e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37165 } } peers { permanent_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 45025 } } peers { permanent_uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42689 } }
I20260806 13:06:53.429754 20787 leader_election.cc:288] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [CANDIDATE]: Term 1 election: Requested vote from peers d4b928cef62841658fa4881246d48c1e (127.13.224.66:37165), 3350e4148f2d4ffbb75f6b9cb2268bb5 (127.13.224.67:42689)
I20260806 13:06:53.429899 20692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344696fbe916417ba36b4640f9faa501" candidate_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5"
I20260806 13:06:53.429934 20493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344696fbe916417ba36b4640f9faa501" candidate_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4b928cef62841658fa4881246d48c1e"
I20260806 13:06:53.429973 20692 raft_consensus.cc:3037] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:53.430035 20493 raft_consensus.cc:3037] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:53.430665 20692 raft_consensus.cc:2454] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b3c5f5635c146a5bc04c6e5a0700259 in term 1.
I20260806 13:06:53.430799 20493 raft_consensus.cc:2454] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b3c5f5635c146a5bc04c6e5a0700259 in term 1.
I20260806 13:06:53.430845 20202 leader_election.cc:302] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3350e4148f2d4ffbb75f6b9cb2268bb5, 3b3c5f5635c146a5bc04c6e5a0700259; no voters:
I20260806 13:06:53.430979 20787 raft_consensus.cc:2781] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:53.431107 20787 raft_consensus.cc:686] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [term 1 LEADER]: Becoming Leader. State: Replica: 3b3c5f5635c146a5bc04c6e5a0700259, State: Running, Role: LEADER
I20260806 13:06:53.431178 20787 consensus_queue.cc:237] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b928cef62841658fa4881246d48c1e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37165 } } peers { permanent_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 45025 } } peers { permanent_uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42689 } }
I20260806 13:06:53.431953 20092 catalog_manager.cc:5495] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b3c5f5635c146a5bc04c6e5a0700259 (127.13.224.65). New cstate: current_term: 1 leader_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4b928cef62841658fa4881246d48c1e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b3c5f5635c146a5bc04c6e5a0700259" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 45025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42689 } health_report { overall_health: UNKNOWN } } }
I20260806 13:06:53.443050 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.440874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37220 (local address 127.13.224.65:45025)
0806 13:06:53.440930 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.440932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.440940 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:06:53.440975 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.440975 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.440990 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.441048 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:06:53.441172 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.441600 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.442379 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:53.442552 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:53.442600 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:53.442754 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:06:53.442847 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:06:53.442916 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.442953 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.442971 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 13:06:53.444597 20692 raft_consensus.cc:1261] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5 [term 1 FOLLOWER]: Refusing update from remote peer 3b3c5f5635c146a5bc04c6e5a0700259: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:53.444595 20493 raft_consensus.cc:1261] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e [term 1 FOLLOWER]: Refusing update from remote peer 3b3c5f5635c146a5bc04c6e5a0700259: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:53.444957 20792 consensus_queue.cc:1035] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3350e4148f2d4ffbb75f6b9cb2268bb5" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:53.445083 20787 consensus_queue.cc:1035] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4b928cef62841658fa4881246d48c1e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:53.449872 20092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:53.449963 20092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:53.450680 20092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 713afb16224b4bb4879d1860158a8126: Sending DeleteTablet for 3 replicas of tablet 344696fbe916417ba36b4640f9faa501
I20260806 13:06:53.450965 20465 tablet_service.cc:1514] Processing DeleteTablet for tablet 344696fbe916417ba36b4640f9faa501 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:53 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:35806
I20260806 13:06:53.450991 20657 tablet_service.cc:1514] Processing DeleteTablet for tablet 344696fbe916417ba36b4640f9faa501 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:53 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:53546
I20260806 13:06:53.451013 20264 tablet_service.cc:1514] Processing DeleteTablet for tablet 344696fbe916417ba36b4640f9faa501 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:53 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:37218
I20260806 13:06:53.451193 20809 tablet_replica.cc:330] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259: stopping tablet replica
I20260806 13:06:53.451304 20809 raft_consensus.cc:2229] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:06:53.451421 20807 tablet_replica.cc:330] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e: stopping tablet replica
I20260806 13:06:53.451431 20809 raft_consensus.cc:2258] T 344696fbe916417ba36b4640f9faa501 P 3b3c5f5635c146a5bc04c6e5a0700259 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:53.451509 20807 raft_consensus.cc:2229] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:53.451509 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 20177
I20260806 13:06:53.451563 20812 tablet_replica.cc:330] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5: stopping tablet replica
I20260806 13:06:53.451596 20807 raft_consensus.cc:2258] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:53.451638 20812 raft_consensus.cc:2229] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:53.455507 20807 ts_tablet_manager.cc:1905] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:53.457074 20807 ts_tablet_manager.cc:1918] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:06:53.457154 20807 log.cc:1196] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/344696fbe916417ba36b4640f9faa501
I20260806 13:06:53.457466 20807 ts_tablet_manager.cc:1939] T 344696fbe916417ba36b4640f9faa501 P d4b928cef62841658fa4881246d48c1e: Deleting consensus metadata
I20260806 13:06:53.458148 20065 catalog_manager.cc:4841] TS d4b928cef62841658fa4881246d48c1e (127.13.224.66:37165): tablet 344696fbe916417ba36b4640f9faa501 (table test-table [id=e49fa65ddf1147d4a5bef385bc2cd50e]) successfully deleted
I20260806 13:06:53.462129 20812 raft_consensus.cc:2258] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:53.462525 20812 ts_tablet_manager.cc:1905] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 13:06:53.463585 20065 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:45025 (error 108)
I20260806 13:06:53.463755 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 20367
I20260806 13:06:53.463876 20812 ts_tablet_manager.cc:1918] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:06:53.463938 20812 log.cc:1196] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/344696fbe916417ba36b4640f9faa501
I20260806 13:06:53.464143 20812 ts_tablet_manager.cc:1939] T 344696fbe916417ba36b4640f9faa501 P 3350e4148f2d4ffbb75f6b9cb2268bb5: Deleting consensus metadata
I20260806 13:06:53.464581 20067 catalog_manager.cc:4841] TS 3350e4148f2d4ffbb75f6b9cb2268bb5 (127.13.224.67:42689): tablet 344696fbe916417ba36b4640f9faa501 (table test-table [id=e49fa65ddf1147d4a5bef385bc2cd50e]) successfully deleted
I20260806 13:06:53.466753 20767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.466617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:45025 (local address 127.0.0.1:37232)
0806 13:06:53.466671 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.466719 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 13:06:53.466874 20065 catalog_manager.cc:4584] TS 3b3c5f5635c146a5bc04c6e5a0700259 (127.13.224.65:45025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 344696fbe916417ba36b4640f9faa501: Network error: Client connection negotiation failed: client connection to 127.13.224.65:45025: connect: Connection refused (error 111)
I20260806 13:06:53.472391 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 20562
I20260806 13:06:53.490556 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 20043
2026-08-06T13:06:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2026 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T13:06:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:53Z Disabled control of system clock
I20260806 13:06:53.520383 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:37303
--webserver_interface=127.13.224.126
--webserver_port=0
--builtin_ntp_servers=127.13.224.84:40595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:37303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 13:06:53.599501 20818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:53.599684 20818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:53.599704 20818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:53.600804 20818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:53.600844 20818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:53.600855 20818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:53.600878 20818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:53.600901 20818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:06:53.602244 20818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:37303
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:37303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:53.602455 20818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:53.602725 20818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:53.607183 20829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:53.607193 20832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:53.607195 20818 server_base.cc:1034] running on GCE node
W20260806 13:06:53.607300 20825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:53.607707 20818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:53.607955 20818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:53.609086 20818 hybrid_clock.cc:648] HybridClock initialized: now 1786021613609079 us; error 27 us; skew 500 ppm
I20260806 13:06:53.610186 20818 webserver.cc:460] Webserver started at http://127.13.224.126:44487/ using document root <none> and password file <none>
I20260806 13:06:53.610404 20818 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:53.610446 20818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:53.610567 20818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:53.611516 20818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "66206fef998041d0bf581958c6cd867f"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "81d7c40ca77d8d79703ff527d92fa54b"
server_key_iv: "72064b4f2fef33eeb72d35f63df8db22"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.611835 20818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "66206fef998041d0bf581958c6cd867f"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "81d7c40ca77d8d79703ff527d92fa54b"
server_key_iv: "72064b4f2fef33eeb72d35f63df8db22"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.612987 20818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:06:53.613608 20843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.613780 20818 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:53.613871 20818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "66206fef998041d0bf581958c6cd867f"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "81d7c40ca77d8d79703ff527d92fa54b"
server_key_iv: "72064b4f2fef33eeb72d35f63df8db22"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.613933 20818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:53.617018 20818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:53.617259 20818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:53.617378 20818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:53.620925 20818 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:37303
I20260806 13:06:53.620959 20917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:37303 every 8 connection(s)
I20260806 13:06:53.621236 20818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:53.621829 20919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:53.623989 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 20818
I20260806 13:06:53.624085 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:53.624282 14209 external_mini_cluster.cc:1428] Setting key abfdee268d57a7535a15df0df3058f61
I20260806 13:06:53.624806 20919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f: Bootstrap starting.
I20260806 13:06:53.625385 20919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:53.625721 20919 log.cc:824] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:53.626858 20924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.624759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43696 (local address 127.13.224.126:37303)
0806 13:06:53.624957 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.624965 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.624996 (+ 31us) server_negotiation.cc:407] Connection header received
0806 13:06:53.625402 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.625411 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.625461 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.625610 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:53.626417 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:53.626421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:53.626422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:53.626427 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:53.626472 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:53.626488 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:53.626585 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.626618 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.626674 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260806 13:06:53.629375 20919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f: No bootstrap required, opened a new log
I20260806 13:06:53.631423 20919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66206fef998041d0bf581958c6cd867f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37303 } }
I20260806 13:06:53.631590 20919 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:53.631635 20919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66206fef998041d0bf581958c6cd867f, State: Initialized, Role: FOLLOWER
I20260806 13:06:53.631771 20919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66206fef998041d0bf581958c6cd867f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37303 } }
I20260806 13:06:53.631862 20919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:53.631901 20919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:53.631948 20919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:53.632795 20919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66206fef998041d0bf581958c6cd867f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37303 } }
I20260806 13:06:53.632892 20919 leader_election.cc:302] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66206fef998041d0bf581958c6cd867f; no voters:
I20260806 13:06:53.633046 20919 leader_election.cc:288] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:53.633107 20929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:53.633272 20929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [term 1 LEADER]: Becoming Leader. State: Replica: 66206fef998041d0bf581958c6cd867f, State: Running, Role: LEADER
I20260806 13:06:53.633311 20919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:53.633360 20929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66206fef998041d0bf581958c6cd867f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37303 } }
I20260806 13:06:53.633692 20930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66206fef998041d0bf581958c6cd867f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66206fef998041d0bf581958c6cd867f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37303 } } }
I20260806 13:06:53.633694 20931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66206fef998041d0bf581958c6cd867f. Latest consensus state: current_term: 1 leader_uuid: "66206fef998041d0bf581958c6cd867f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66206fef998041d0bf581958c6cd867f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37303 } } }
I20260806 13:06:53.633813 20930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:53.633821 20931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:53.634362 20944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:53.634866 20944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:53.636236 20944 catalog_manager.cc:1347] Generated new cluster ID: d944784d6f014c239a521104037b1cc5
I20260806 13:06:53.636287 20944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:06:53.649983 20944 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:53.650177 20944 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:53.655426 20944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f: Generated new TSK 0
I20260806 13:06:53.655593 20944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:06:53.660897 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37303
--builtin_ntp_servers=127.13.224.84:40595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 13:06:53.741118 20961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:53.741355 20961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:53.741389 20961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:53.742596 20961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:53.742640 20961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:53.742676 20961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:06:53.743994 20961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:53.744208 20961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:53.744463 20961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:53.749084 20975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:53.749238 20961 server_base.cc:1034] running on GCE node
W20260806 13:06:53.749181 20971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:53.749076 20970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:53.749517 20961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:53.749735 20961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:53.750869 20961 hybrid_clock.cc:648] HybridClock initialized: now 1786021613750859 us; error 33 us; skew 500 ppm
I20260806 13:06:53.752017 20961 webserver.cc:460] Webserver started at http://127.13.224.65:46001/ using document root <none> and password file <none>
I20260806 13:06:53.752252 20961 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:53.752300 20961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:53.752429 20961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:53.753422 20961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "36f2ce8892b540f0aaec36b6fe3896e9"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "d72c453cbfb210919b517ef9d2578750"
server_key_iv: "73d85d47b81e48b45d1b7ab1849ca96e"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.753742 20961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "36f2ce8892b540f0aaec36b6fe3896e9"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "d72c453cbfb210919b517ef9d2578750"
server_key_iv: "73d85d47b81e48b45d1b7ab1849ca96e"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.754997 20961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:06:53.755653 20984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.755810 20961 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:53.755887 20961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "36f2ce8892b540f0aaec36b6fe3896e9"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "d72c453cbfb210919b517ef9d2578750"
server_key_iv: "73d85d47b81e48b45d1b7ab1849ca96e"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.755950 20961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:53.759083 20961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:53.759320 20961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:53.759433 20961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:53.759624 20961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:53.759893 20961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:53.759925 20961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.759953 20961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:53.759975 20961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.765283 20961 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:40449
I20260806 13:06:53.765344 21155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:40449 every 8 connection(s)
I20260806 13:06:53.765601 20961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:06:53.765947 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 20961
I20260806 13:06:53.766062 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:06:53.766258 14209 external_mini_cluster.cc:1428] Setting key fd066f1695983abbb17b54d3f87dad7a
I20260806 13:06:53.767273 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37303
--builtin_ntp_servers=127.13.224.84:40595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 13:06:53.768594 20924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.766326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:34491 (local address 127.13.224.126:37303)
0806 13:06:53.766368 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.766370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.766609 (+ 239us) server_negotiation.cc:407] Connection header received
0806 13:06:53.767159 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.767160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.767165 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.767199 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:53.768382 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:53.768383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:53.768383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:53.768387 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:53.768402 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:53.768406 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:53.768449 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.768534 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.768545 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:06:53.768806 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.766327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37303 (local address 127.13.224.65:34491)
0806 13:06:53.766505 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.766542 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:53.767063 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:53.767273 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:53.767277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:53.767287 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:53.768229 (+ 942us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:06:53.768230 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:53.768266 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:06:53.768299 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:53.768467 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:53.768471 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:53.768474 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:06:53.768544 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 13:06:53.768603 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260806 13:06:53.769232 21156 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37303
I20260806 13:06:53.769307 21156 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:53.769441 21156 heartbeater.cc:511] Master 127.13.224.126:37303 requested a full tablet report, sending...
I20260806 13:06:53.769901 20863 ts_manager.cc:194] Registered new tserver with Master: 36f2ce8892b540f0aaec36b6fe3896e9 (127.13.224.65:40449)
W20260806 13:06:53.847545 21160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:53.847723 21160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:53.847743 21160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:53.848863 21160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:53.848899 21160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:53.848927 21160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:06:53.850273 21160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:53.850477 21160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:53.850732 21160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:53.855810 21169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:53.855947 21174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:53.855996 21160 server_base.cc:1034] running on GCE node
W20260806 13:06:53.855862 21168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:53.856200 21160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:53.856427 21160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:53.857561 21160 hybrid_clock.cc:648] HybridClock initialized: now 1786021613857549 us; error 31 us; skew 500 ppm
I20260806 13:06:53.858709 21160 webserver.cc:460] Webserver started at http://127.13.224.66:45299/ using document root <none> and password file <none>
I20260806 13:06:53.858937 21160 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:53.858986 21160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:53.859112 21160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:53.860144 21160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "d67f70339f9d4487a89e023b32759680"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "db29b8244b83156311e9cb1a16586fb8"
server_key_iv: "b00b181a86b87bcaa1a1328f0704cdf1"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.860468 21160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "d67f70339f9d4487a89e023b32759680"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "db29b8244b83156311e9cb1a16586fb8"
server_key_iv: "b00b181a86b87bcaa1a1328f0704cdf1"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.861620 21160 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:06:53.862275 21185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.862422 21160 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:53.862502 21160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "d67f70339f9d4487a89e023b32759680"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "db29b8244b83156311e9cb1a16586fb8"
server_key_iv: "b00b181a86b87bcaa1a1328f0704cdf1"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.862565 21160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:53.865592 21160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:53.865823 21160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:53.865940 21160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:53.866142 21160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:53.866425 21160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:53.866458 21160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.866484 21160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:53.866508 21160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.871793 21160 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:32901
I20260806 13:06:53.871825 21344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:32901 every 8 connection(s)
I20260806 13:06:53.872092 21160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:06:53.872627 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 21160
I20260806 13:06:53.872733 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:06:53.872929 14209 external_mini_cluster.cc:1428] Setting key f103920e61a93f493bc3e1303c724592
I20260806 13:06:53.873761 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37303
--builtin_ntp_servers=127.13.224.84:40595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 13:06:53.874826 20924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.872703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:56781 (local address 127.13.224.126:37303)
0806 13:06:53.872744 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.872746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.872989 (+ 243us) server_negotiation.cc:407] Connection header received
0806 13:06:53.873405 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.873406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.873413 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.873463 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:53.874583 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:53.874584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:53.874585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:53.874589 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:53.874605 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:53.874609 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:53.874658 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.874731 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.874744 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:06:53.875003 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.872712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37303 (local address 127.13.224.66:56781)
0806 13:06:53.872882 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.872912 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:53.873315 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:53.873526 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:53.873529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:53.873539 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:53.874447 (+ 908us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:06:53.874448 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:53.874474 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:06:53.874499 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:53.874661 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:53.874666 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:53.874670 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:06:53.874737 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 13:06:53.874814 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260806 13:06:53.875376 21345 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37303
I20260806 13:06:53.875470 21345 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:53.875607 21345 heartbeater.cc:511] Master 127.13.224.126:37303 requested a full tablet report, sending...
I20260806 13:06:53.875936 20863 ts_manager.cc:194] Registered new tserver with Master: d67f70339f9d4487a89e023b32759680 (127.13.224.66:32901)
W20260806 13:06:53.953730 21351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:53.953920 21351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:53.953941 21351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:53.955088 21351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:53.955127 21351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:53.955150 21351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:06:53.956432 21351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:53.956612 21351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:53.956856 21351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:53.961645 21358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:53.961558 21360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:53.961558 21356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:53.961673 21351 server_base.cc:1034] running on GCE node
I20260806 13:06:53.962059 21351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:53.962296 21351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:53.963428 21351 hybrid_clock.cc:648] HybridClock initialized: now 1786021613963420 us; error 27 us; skew 500 ppm
I20260806 13:06:53.964715 21351 webserver.cc:460] Webserver started at http://127.13.224.67:38927/ using document root <none> and password file <none>
I20260806 13:06:53.964953 21351 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:53.964996 21351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:53.965126 21351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:53.966252 21351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "99641dc3a37940718b35d24b98072656"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "376ddf778148c2a3ab8667800db45d94"
server_key_iv: "5abe60bcd5d2546e9a097d28c1197e68"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.966573 21351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "99641dc3a37940718b35d24b98072656"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "376ddf778148c2a3ab8667800db45d94"
server_key_iv: "5abe60bcd5d2546e9a097d28c1197e68"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.967881 21351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:06:53.968508 21368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.968653 21351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:53.968734 21351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "99641dc3a37940718b35d24b98072656"
format_stamp: "Formatted at 2026-08-06 13:06:53 on dist-test-slave-q9h9"
server_key: "376ddf778148c2a3ab8667800db45d94"
server_key_iv: "5abe60bcd5d2546e9a097d28c1197e68"
server_key_version: "encryptionkey@0"
I20260806 13:06:53.968802 21351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:53.971983 21351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:53.972213 21351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:53.972327 21351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:53.972501 21351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:53.972782 21351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:53.972816 21351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.972842 21351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:53.972867 21351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:53.978250 21351 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:40863
I20260806 13:06:53.978312 21543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:40863 every 8 connection(s)
I20260806 13:06:53.978611 21351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:06:53.981127 20924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.979209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:47053 (local address 127.13.224.126:37303)
0806 13:06:53.979258 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.979260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.979481 (+ 221us) server_negotiation.cc:407] Connection header received
0806 13:06:53.979925 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.979926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.979932 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.979968 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:53.980954 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:53.980955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:53.980955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:53.980959 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:53.980974 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:53.980978 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:53.981018 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.981063 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.981074 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:06:53.981290 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.979177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37303 (local address 127.13.224.67:47053)
0806 13:06:53.979362 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.979400 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:53.979830 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:53.980024 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:53.980027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:53.980034 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:53.980824 (+ 790us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:06:53.980825 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:53.980851 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:06:53.980877 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:53.981018 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:53.981020 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:53.981023 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:06:53.981068 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 13:06:53.981112 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260806 13:06:53.981707 21545 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37303
I20260806 13:06:53.981783 21545 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:53.981920 21545 heartbeater.cc:511] Master 127.13.224.126:37303 requested a full tablet report, sending...
I20260806 13:06:53.982271 20863 ts_manager.cc:194] Registered new tserver with Master: 99641dc3a37940718b35d24b98072656 (127.13.224.67:40863)
I20260806 13:06:53.988662 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 21351
I20260806 13:06:53.988757 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:06:53.988926 14209 external_mini_cluster.cc:1428] Setting key 1d47f55dab62e88981ac4daa279e77be
I20260806 13:06:53.989665 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:06:53.991756 20924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.990934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43710 (local address 127.13.224.126:37303)
0806 13:06:53.990989 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.990991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.991081 (+ 90us) server_negotiation.cc:407] Connection header received
0806 13:06:53.991119 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.991119 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.991124 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.991165 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:53.991593 (+ 428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:53.991594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:53.991595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:53.991599 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:53.991612 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:53.991615 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:53.991653 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.991682 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.991693 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 13:06:53.993052 20863 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:06:53.994608 20863 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:06:53.998272 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.996620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34308 (local address 127.13.224.67:40863)
0806 13:06:53.996780 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.996785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.996909 (+ 124us) server_negotiation.cc:407] Connection header received
0806 13:06:53.996954 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.996957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.996965 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.997050 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:53.997948 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:53.997951 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:53.997951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:53.997955 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:53.997984 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:53.997997 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:53.998078 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.998177 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.998192 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260806 13:06:53.998276 21556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.996453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:40449 (local address 127.0.0.1:58768)
0806 13:06:53.996795 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.996832 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:53.996900 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:53.997270 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:53.997271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:53.997273 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:53.997812 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:06:53.997812 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:53.997817 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:06:53.997827 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:53.998086 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:53.998093 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:53.998096 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:06:53.998161 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 13:06:53.998173 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":70,"threads_started":1}
I20260806 13:06:53.998279 21560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.996474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:32901 (local address 127.0.0.1:44306)
0806 13:06:53.996990 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.996997 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:53.997038 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:53.997514 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:53.997515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:53.997517 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:53.997809 (+ 292us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:06:53.997809 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:53.997816 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:06:53.997827 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:53.998106 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:53.998107 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:53.998108 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 13:06:53.998164 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 13:06:53.998177 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":87,"threads_started":1}
I20260806 13:06:53.998519 21564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.996626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58768 (local address 127.13.224.65:40449)
0806 13:06:53.996807 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.996811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.996895 (+ 84us) server_negotiation.cc:407] Connection header received
0806 13:06:53.996963 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.996966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.996975 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.997050 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:53.997912 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:53.997922 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:53.997922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:53.997927 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:53.997959 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:53.997988 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:53.998087 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.998419 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.998437 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":73,"threads_started":1}
I20260806 13:06:53.998281 21562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.996453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:40863 (local address 127.0.0.1:34308)
0806 13:06:53.996856 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:53.996864 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:53.996908 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:53.997061 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:53.997064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:53.997071 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:53.997544 (+ 473us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:06:53.997544 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:53.997561 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:06:53.997583 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:53.998102 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:53.998103 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:53.998103 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 13:06:53.998160 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 13:06:53.998175 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":4,"thread_start_us":63,"threads_started":1}
I20260806 13:06:53.998728 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:53.996691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44306 (local address 127.13.224.66:32901)
0806 13:06:53.996852 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:53.996855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:53.997394 (+ 539us) server_negotiation.cc:407] Connection header received
0806 13:06:53.997425 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:53.997427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:53.997435 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:53.997499 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:53.997912 (+ 413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:53.997922 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:53.997923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:53.997926 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:53.997959 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:53.997985 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:53.998084 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:53.998621 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 13:06:53.998639 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260806 13:06:53.999156 21055 tablet_service.cc:1467] Processing CreateTablet for tablet fad75174eb834da9883e1cab2b679485 (DEFAULT_TABLE table=test-table [id=9959fb184def4dd8a019d40dd161a9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:53.999265 21443 tablet_service.cc:1467] Processing CreateTablet for tablet fad75174eb834da9883e1cab2b679485 (DEFAULT_TABLE table=test-table [id=9959fb184def4dd8a019d40dd161a9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:53.999318 21247 tablet_service.cc:1467] Processing CreateTablet for tablet fad75174eb834da9883e1cab2b679485 (DEFAULT_TABLE table=test-table [id=9959fb184def4dd8a019d40dd161a9df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:06:53.999544 21443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fad75174eb834da9883e1cab2b679485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:53.999545 21055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fad75174eb834da9883e1cab2b679485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:53.999671 21247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fad75174eb834da9883e1cab2b679485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:54.002422 21566 tablet_bootstrap.cc:492] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680: Bootstrap starting.
I20260806 13:06:54.002740 21569 tablet_bootstrap.cc:492] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9: Bootstrap starting.
I20260806 13:06:54.003051 21566 tablet_bootstrap.cc:654] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:54.003317 21566 log.cc:824] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:54.003366 21569 tablet_bootstrap.cc:654] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:54.003661 21569 log.cc:824] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:54.003968 21568 tablet_bootstrap.cc:492] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656: Bootstrap starting.
I20260806 13:06:54.004164 21566 tablet_bootstrap.cc:492] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680: No bootstrap required, opened a new log
I20260806 13:06:54.004240 21566 ts_tablet_manager.cc:1397] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:06:54.004400 21569 tablet_bootstrap.cc:492] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9: No bootstrap required, opened a new log
I20260806 13:06:54.004463 21569 ts_tablet_manager.cc:1397] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:06:54.004542 21568 tablet_bootstrap.cc:654] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:54.004801 21568 log.cc:824] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:54.006470 21568 tablet_bootstrap.cc:492] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656: No bootstrap required, opened a new log
I20260806 13:06:54.006528 21568 ts_tablet_manager.cc:1397] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 13:06:54.006923 21566 raft_consensus.cc:348] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99641dc3a37940718b35d24b98072656" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40863 } } peers { permanent_uuid: "36f2ce8892b540f0aaec36b6fe3896e9" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40449 } } peers { permanent_uuid: "d67f70339f9d4487a89e023b32759680" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 32901 } }
I20260806 13:06:54.006923 21569 raft_consensus.cc:348] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99641dc3a37940718b35d24b98072656" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40863 } } peers { permanent_uuid: "36f2ce8892b540f0aaec36b6fe3896e9" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40449 } } peers { permanent_uuid: "d67f70339f9d4487a89e023b32759680" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 32901 } }
I20260806 13:06:54.007087 21569 raft_consensus.cc:374] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:54.007087 21566 raft_consensus.cc:374] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:54.007123 21569 raft_consensus.cc:729] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f2ce8892b540f0aaec36b6fe3896e9, State: Initialized, Role: FOLLOWER
I20260806 13:06:54.007123 21566 raft_consensus.cc:729] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d67f70339f9d4487a89e023b32759680, State: Initialized, Role: FOLLOWER
I20260806 13:06:54.007241 21566 consensus_queue.cc:260] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99641dc3a37940718b35d24b98072656" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40863 } } peers { permanent_uuid: "36f2ce8892b540f0aaec36b6fe3896e9" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40449 } } peers { permanent_uuid: "d67f70339f9d4487a89e023b32759680" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 32901 } }
I20260806 13:06:54.007303 21569 consensus_queue.cc:260] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99641dc3a37940718b35d24b98072656" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40863 } } peers { permanent_uuid: "36f2ce8892b540f0aaec36b6fe3896e9" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40449 } } peers { permanent_uuid: "d67f70339f9d4487a89e023b32759680" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 32901 } }
I20260806 13:06:54.007501 21566 ts_tablet_manager.cc:1428] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:06:54.007520 21569 ts_tablet_manager.cc:1428] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:06:54.007617 21345 heartbeater.cc:503] Master 127.13.224.126:37303 was elected leader, sending a full tablet report...
I20260806 13:06:54.007623 21156 heartbeater.cc:503] Master 127.13.224.126:37303 was elected leader, sending a full tablet report...
I20260806 13:06:54.008715 21568 raft_consensus.cc:348] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99641dc3a37940718b35d24b98072656" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40863 } } peers { permanent_uuid: "36f2ce8892b540f0aaec36b6fe3896e9" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40449 } } peers { permanent_uuid: "d67f70339f9d4487a89e023b32759680" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 32901 } }
I20260806 13:06:54.008834 21568 raft_consensus.cc:374] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:54.008853 21568 raft_consensus.cc:729] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99641dc3a37940718b35d24b98072656, State: Initialized, Role: FOLLOWER
I20260806 13:06:54.008937 21568 consensus_queue.cc:260] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99641dc3a37940718b35d24b98072656" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40863 } } peers { permanent_uuid: "36f2ce8892b540f0aaec36b6fe3896e9" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40449 } } peers { permanent_uuid: "d67f70339f9d4487a89e023b32759680" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 32901 } }
I20260806 13:06:54.009132 21568 ts_tablet_manager.cc:1428] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:06:54.009184 21545 heartbeater.cc:503] Master 127.13.224.126:37303 was elected leader, sending a full tablet report...
W20260806 13:06:54.016225 21158 tablet.cc:2369] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:06:54.110735 21577 raft_consensus.cc:482] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:06:54.110863 21577 raft_consensus.cc:504] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99641dc3a37940718b35d24b98072656" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40863 } } peers { permanent_uuid: "36f2ce8892b540f0aaec36b6fe3896e9" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40449 } } peers { permanent_uuid: "d67f70339f9d4487a89e023b32759680" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 32901 } }
I20260806 13:06:54.111232 21577 leader_election.cc:288] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99641dc3a37940718b35d24b98072656 (127.13.224.67:40863), 36f2ce8892b540f0aaec36b6fe3896e9 (127.13.224.65:40449)
I20260806 13:06:54.112418 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:54.111415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:40863 (local address 127.13.224.66:59399)
0806 13:06:54.111461 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:54.111472 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:54.111515 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:54.111677 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:54.111678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:54.111680 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:54.112084 (+ 404us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:06:54.112084 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:54.112092 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:06:54.112104 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:54.112302 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:54.112302 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:54.112303 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 13:06:54.112336 (+ 33us) client_negotiation.cc:240] Negotiation successful
0806 13:06:54.112349 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 13:06:54.112433 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:54.111512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:59399 (local address 127.13.224.67:40863)
0806 13:06:54.111561 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:54.111563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:54.111572 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:06:54.111599 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:54.111600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:54.111606 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:54.111657 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:54.112185 (+ 528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:54.112187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:54.112187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:54.112192 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:54.112212 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:54.112216 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:54.112291 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:54.112351 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 13:06:54.112365 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:06:54.112478 21564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:54.111527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:39689 (local address 127.13.224.65:40449)
0806 13:06:54.111569 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:54.111571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:54.111671 (+ 100us) server_negotiation.cc:407] Connection header received
0806 13:06:54.111704 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:54.111706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:54.111711 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:54.111760 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:54.112275 (+ 515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:54.112277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:54.112277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:54.112282 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:54.112299 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:54.112304 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:54.112358 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:54.112408 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 13:06:54.112420 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:06:54.112519 21581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:54.111419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:40449 (local address 127.13.224.66:39689)
0806 13:06:54.111606 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:54.111614 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:54.111662 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:54.111773 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:54.111774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:54.111776 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:54.112181 (+ 405us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:06:54.112182 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:54.112187 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:06:54.112196 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:54.112359 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:54.112359 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:54.112360 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 13:06:54.112399 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 13:06:54.112409 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":13,"thread_start_us":106,"threads_started":1}
I20260806 13:06:54.112824 21474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fad75174eb834da9883e1cab2b679485" candidate_uuid: "d67f70339f9d4487a89e023b32759680" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99641dc3a37940718b35d24b98072656" is_pre_election: true
I20260806 13:06:54.112864 21079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fad75174eb834da9883e1cab2b679485" candidate_uuid: "d67f70339f9d4487a89e023b32759680" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f2ce8892b540f0aaec36b6fe3896e9" is_pre_election: true
I20260806 13:06:54.112985 21474 raft_consensus.cc:2454] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d67f70339f9d4487a89e023b32759680 in term 0.
I20260806 13:06:54.112985 21079 raft_consensus.cc:2454] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d67f70339f9d4487a89e023b32759680 in term 0.
I20260806 13:06:54.113227 21187 leader_election.cc:302] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99641dc3a37940718b35d24b98072656, d67f70339f9d4487a89e023b32759680; no voters:
I20260806 13:06:54.113381 21577 raft_consensus.cc:2781] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:06:54.113435 21577 raft_consensus.cc:482] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:06:54.113456 21577 raft_consensus.cc:3037] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:54.114198 21577 raft_consensus.cc:504] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99641dc3a37940718b35d24b98072656" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40863 } } peers { permanent_uuid: "36f2ce8892b540f0aaec36b6fe3896e9" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40449 } } peers { permanent_uuid: "d67f70339f9d4487a89e023b32759680" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 32901 } }
I20260806 13:06:54.114318 21577 leader_election.cc:288] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [CANDIDATE]: Term 1 election: Requested vote from peers 99641dc3a37940718b35d24b98072656 (127.13.224.67:40863), 36f2ce8892b540f0aaec36b6fe3896e9 (127.13.224.65:40449)
I20260806 13:06:54.114481 21474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fad75174eb834da9883e1cab2b679485" candidate_uuid: "d67f70339f9d4487a89e023b32759680" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99641dc3a37940718b35d24b98072656"
I20260806 13:06:54.114485 21079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fad75174eb834da9883e1cab2b679485" candidate_uuid: "d67f70339f9d4487a89e023b32759680" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f2ce8892b540f0aaec36b6fe3896e9"
I20260806 13:06:54.114562 21079 raft_consensus.cc:3037] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:54.114562 21474 raft_consensus.cc:3037] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:54.115433 21474 raft_consensus.cc:2454] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d67f70339f9d4487a89e023b32759680 in term 1.
I20260806 13:06:54.115433 21079 raft_consensus.cc:2454] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d67f70339f9d4487a89e023b32759680 in term 1.
I20260806 13:06:54.115587 21187 leader_election.cc:302] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99641dc3a37940718b35d24b98072656, d67f70339f9d4487a89e023b32759680; no voters:
I20260806 13:06:54.115687 21577 raft_consensus.cc:2781] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:54.115866 21577 raft_consensus.cc:686] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [term 1 LEADER]: Becoming Leader. State: Replica: d67f70339f9d4487a89e023b32759680, State: Running, Role: LEADER
I20260806 13:06:54.115957 21577 consensus_queue.cc:237] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99641dc3a37940718b35d24b98072656" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40863 } } peers { permanent_uuid: "36f2ce8892b540f0aaec36b6fe3896e9" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40449 } } peers { permanent_uuid: "d67f70339f9d4487a89e023b32759680" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 32901 } }
I20260806 13:06:54.116771 20862 catalog_manager.cc:5495] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 reported cstate change: term changed from 0 to 1, leader changed from <none> to d67f70339f9d4487a89e023b32759680 (127.13.224.66). New cstate: current_term: 1 leader_uuid: "d67f70339f9d4487a89e023b32759680" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99641dc3a37940718b35d24b98072656" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36f2ce8892b540f0aaec36b6fe3896e9" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d67f70339f9d4487a89e023b32759680" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 32901 } health_report { overall_health: HEALTHY } } }
W20260806 13:06:54.122538 21346 tablet.cc:2369] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:06:54.141625 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:54.140936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44322 (local address 127.13.224.66:32901)
0806 13:06:54.140981 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:54.140983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:54.140989 (+ 6us) server_negotiation.cc:407] Connection header received
0806 13:06:54.141018 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:54.141019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:54.141030 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:54.141076 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:54.141479 (+ 403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:54.141480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:54.141481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:54.141484 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:54.141498 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:54.141501 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:54.141540 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:54.141567 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 13:06:54.141578 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:06:54.143256 21079 raft_consensus.cc:1261] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9 [term 1 FOLLOWER]: Refusing update from remote peer d67f70339f9d4487a89e023b32759680: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:54.143405 21474 raft_consensus.cc:1261] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656 [term 1 FOLLOWER]: Refusing update from remote peer d67f70339f9d4487a89e023b32759680: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:06:54.143555 21577 consensus_queue.cc:1035] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36f2ce8892b540f0aaec36b6fe3896e9" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:54.143693 21582 consensus_queue.cc:1035] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99641dc3a37940718b35d24b98072656" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:06:54.146175 21592 mvcc.cc:205] Tried to move back new op lower bound from 7315544531528429568 to 7315544531419480064. Current Snapshot: MvccSnapshot[applied={T|T < 7315544531528429568}]
I20260806 13:06:54.146853 21591 mvcc.cc:205] Tried to move back new op lower bound from 7315544531528429568 to 7315544531419480064. Current Snapshot: MvccSnapshot[applied={T|T < 7315544531528429568 or (T in {7315544531528429568})}]
I20260806 13:06:54.147895 20862 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:54.147972 20862 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:06:54.148629 20862 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66206fef998041d0bf581958c6cd867f: Sending DeleteTablet for 3 replicas of tablet fad75174eb834da9883e1cab2b679485
I20260806 13:06:54.148933 21443 tablet_service.cc:1514] Processing DeleteTablet for tablet fad75174eb834da9883e1cab2b679485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:54 UTC) from {username='slave'} at 127.0.0.1:34308
I20260806 13:06:54.148974 21055 tablet_service.cc:1514] Processing DeleteTablet for tablet fad75174eb834da9883e1cab2b679485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:54 UTC) from {username='slave'} at 127.0.0.1:58768
I20260806 13:06:54.149205 21605 tablet_replica.cc:330] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9: stopping tablet replica
I20260806 13:06:54.149290 21603 tablet_replica.cc:330] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656: stopping tablet replica
I20260806 13:06:54.149386 21605 raft_consensus.cc:2229] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:54.149392 21603 raft_consensus.cc:2229] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:06:54.149386 21247 tablet_service.cc:1514] Processing DeleteTablet for tablet fad75174eb834da9883e1cab2b679485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:06:54 UTC) from {username='slave'} at 127.0.0.1:44306
I20260806 13:06:54.149492 21603 raft_consensus.cc:2258] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:54.149570 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 20961
I20260806 13:06:54.149600 21605 raft_consensus.cc:2258] T fad75174eb834da9883e1cab2b679485 P 36f2ce8892b540f0aaec36b6fe3896e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:06:54.149951 21603 ts_tablet_manager.cc:1905] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:54.151592 21603 ts_tablet_manager.cc:1918] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:06:54.151657 21603 log.cc:1196] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/fad75174eb834da9883e1cab2b679485
I20260806 13:06:54.151854 21603 ts_tablet_manager.cc:1939] T fad75174eb834da9883e1cab2b679485 P 99641dc3a37940718b35d24b98072656: Deleting consensus metadata
I20260806 13:06:54.152210 20845 catalog_manager.cc:4841] TS 99641dc3a37940718b35d24b98072656 (127.13.224.67:40863): tablet fad75174eb834da9883e1cab2b679485 (table test-table [id=9959fb184def4dd8a019d40dd161a9df]) successfully deleted
I20260806 13:06:54.160527 21606 tablet_replica.cc:330] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680: stopping tablet replica
I20260806 13:06:54.160650 21606 raft_consensus.cc:2229] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:06:54.160807 21606 raft_consensus.cc:2258] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 13:06:54.164238 20847 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:40449 (error 108)
I20260806 13:06:54.164556 21606 ts_tablet_manager.cc:1905] T fad75174eb834da9883e1cab2b679485 P d67f70339f9d4487a89e023b32759680: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:06:54.164733 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 21160
I20260806 13:06:54.170111 21562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:54.169477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:40449 (local address 127.0.0.1:58782)
0806 13:06:54.169985 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:54.170070 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:40449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483}
W20260806 13:06:54.170270 20847 catalog_manager.cc:4584] TS 36f2ce8892b540f0aaec36b6fe3896e9 (127.13.224.65:40449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fad75174eb834da9883e1cab2b679485: Network error: Client connection negotiation failed: client connection to 127.13.224.65:40449: connect: Connection refused (error 111)
I20260806 13:06:54.174607 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 21351
I20260806 13:06:54.174621 21562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:54.174515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:32901 (local address 127.0.0.1:44332)
0806 13:06:54.174567 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:54.174596 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.66:32901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 13:06:54.184790 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 20818
2026-08-06T13:06:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (697 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 13:06:54.210865 14209 mini_oidc.cc:152] Starting JWKS server
I20260806 13:06:54.212040 14209 webserver.cc:460] Webserver started at https://127.0.0.1:39049/ using document root <none> and password file <none>
I20260806 13:06:54.212127 14209 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 13:06:54.212286 14209 webserver.cc:460] Webserver started at http://127.0.0.1:34185/ using document root <none> and password file <none>
2026-08-06T13:06:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:54Z Disabled control of system clock
I20260806 13:06:54.220417 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:38211
--webserver_interface=127.13.224.126
--webserver_port=0
--builtin_ntp_servers=127.13.224.84:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:38211
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39049/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/testchainca.cert with env {}
W20260806 13:06:54.303326 21615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:54.303522 21615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:54.303543 21615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:54.304699 21615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:54.304739 21615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:54.304750 21615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:54.304761 21615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:54.304772 21615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 13:06:54.304785 21615 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 13:06:54.304796 21615 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39049/jwks.json
I20260806 13:06:54.306321 21615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:38211
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:38211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39049/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:54.306507 21615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:54.306665 21615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 13:06:54.311273 21615 server_base.cc:1034] running on GCE node
W20260806 13:06:54.311241 21622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:54.311379 21625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:54.311267 21623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:54.311686 21615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:54.311949 21615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:54.313090 21615 hybrid_clock.cc:648] HybridClock initialized: now 1786021614313077 us; error 31 us; skew 500 ppm
I20260806 13:06:54.314248 21615 webserver.cc:460] Webserver started at http://127.13.224.126:37593/ using document root <none> and password file <none>
I20260806 13:06:54.314463 21615 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:54.314512 21615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:54.314644 21615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:54.315763 21615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "cc1559b25b4c4a8e850b805d5654fbb0"
format_stamp: "Formatted at 2026-08-06 13:06:54 on dist-test-slave-q9h9"
server_key: "39b01d58f622eb7ece51cf100f2fcaa4"
server_key_iv: "8cd0856add15471be7761913286c23fd"
server_key_version: "encryptionkey@0"
I20260806 13:06:54.316089 21615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "cc1559b25b4c4a8e850b805d5654fbb0"
format_stamp: "Formatted at 2026-08-06 13:06:54 on dist-test-slave-q9h9"
server_key: "39b01d58f622eb7ece51cf100f2fcaa4"
server_key_iv: "8cd0856add15471be7761913286c23fd"
server_key_version: "encryptionkey@0"
I20260806 13:06:54.317420 21615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:06:54.318068 21632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:54.318221 21615 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:54.318295 21615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "cc1559b25b4c4a8e850b805d5654fbb0"
format_stamp: "Formatted at 2026-08-06 13:06:54 on dist-test-slave-q9h9"
server_key: "39b01d58f622eb7ece51cf100f2fcaa4"
server_key_iv: "8cd0856add15471be7761913286c23fd"
server_key_version: "encryptionkey@0"
I20260806 13:06:54.318363 21615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:54.335729 21615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:54.335999 21615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:54.336127 21615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:54.339828 21615 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:38211
I20260806 13:06:54.339903 21709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:38211 every 8 connection(s)
I20260806 13:06:54.340193 21615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:54.341070 21710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:54.343384 21710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0: Bootstrap starting.
I20260806 13:06:54.343978 21710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:54.344223 21710 log.cc:824] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:54.344501 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 21615
I20260806 13:06:54.344611 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:54.344776 14209 external_mini_cluster.cc:1428] Setting key 139a3772dc08c154e47be53a2505e08e
I20260806 13:06:54.344925 21710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0: No bootstrap required, opened a new log
I20260806 13:06:54.347646 21710 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc1559b25b4c4a8e850b805d5654fbb0" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 38211 } }
I20260806 13:06:54.347805 21710 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:54.347836 21710 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1559b25b4c4a8e850b805d5654fbb0, State: Initialized, Role: FOLLOWER
I20260806 13:06:54.347950 21710 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc1559b25b4c4a8e850b805d5654fbb0" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 38211 } }
I20260806 13:06:54.348028 21710 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:54.348068 21710 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:54.348109 21710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:54.348613 21710 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc1559b25b4c4a8e850b805d5654fbb0" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 38211 } }
I20260806 13:06:54.348709 21710 leader_election.cc:302] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc1559b25b4c4a8e850b805d5654fbb0; no voters:
I20260806 13:06:54.348865 21710 leader_election.cc:288] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:54.348891 21717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:54.349012 21717 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [term 1 LEADER]: Becoming Leader. State: Replica: cc1559b25b4c4a8e850b805d5654fbb0, State: Running, Role: LEADER
I20260806 13:06:54.349095 21717 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc1559b25b4c4a8e850b805d5654fbb0" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 38211 } }
I20260806 13:06:54.349135 21710 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:54.349402 21717 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc1559b25b4c4a8e850b805d5654fbb0. Latest consensus state: current_term: 1 leader_uuid: "cc1559b25b4c4a8e850b805d5654fbb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc1559b25b4c4a8e850b805d5654fbb0" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 38211 } } }
I20260806 13:06:54.349467 21717 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:54.349790 21719 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc1559b25b4c4a8e850b805d5654fbb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc1559b25b4c4a8e850b805d5654fbb0" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 38211 } } }
I20260806 13:06:54.349920 21719 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0 [sys.catalog]: This master's current role is: LEADER
W20260806 13:06:54.350865 21739 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:06:54.350948 21739 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:06:54.350863 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:54.345189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39282 (local address 127.13.224.126:38211)
0806 13:06:54.345424 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:54.345433 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:54.345471 (+ 38us) server_negotiation.cc:407] Connection header received
0806 13:06:54.346232 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:54.346242 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:54.346320 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:54.346468 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:54.346611 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.347937 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.348749 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.349339 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.349450 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:54.350145 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:54.350149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:54.350149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:54.350342 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:54.350384 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:54.350399 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:54.350550 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:54.350582 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 13:06:54.350657 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
I20260806 13:06:54.351387 21723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:54.351554 21723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:54.351904 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:06:54.353003 21723 catalog_manager.cc:1347] Generated new cluster ID: 520c9653e17242caae45f3887da10a39
I20260806 13:06:54.353053 21723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:06:54.358186 21723 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:54.358759 21723 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:54.361162 21723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc1559b25b4c4a8e850b805d5654fbb0: Generated new TSK 0
I20260806 13:06:54.361315 21723 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 13:06:54.386286 21756 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 13:06:54.386430 21756 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 13:06:54.386610 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:54.368922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39292 (local address 127.13.224.126:38211)
0806 13:06:54.368965 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:54.368967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:54.369108 (+ 141us) server_negotiation.cc:407] Connection header received
0806 13:06:54.369200 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:54.369201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:54.369211 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:54.369274 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0806 13:06:54.369404 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.369852 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.370970 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.371158 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.371204 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:54.371398 (+ 194us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 13:06:54.386380 (+ 14982us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39049/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 13:06:54.386546 (+ 166us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39292: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39049/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 13:06:54.386674 21716 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39292: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39049/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 13:06:54.387204 21752 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:06:54.368865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:38211 (local address 127.0.0.1:39292)
0806 13:06:54.369035 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:54.369046 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:54.369114 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:54.369293 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:54.369294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:54.369296 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 13:06:54.369347 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:54.369348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.369926 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.369927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:54.370901 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:54.370904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.371226 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.371227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:54.371286 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:54.371309 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 13:06:54.386605 (+ 15296us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39049/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 13:06:54.386819 (+ 214us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.224.126:38211: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39049/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.224.126:38211: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39049/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 13:06:54.387689 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 21615
2026-08-06T13:06:54Z chronyd exiting
I20260806 13:06:54.433022 14209 test_util.cc:175] -----------------------------------------------
I20260806 13:06:54.433100 14209 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786021591455662-14209-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (223 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 13:06:54.433972 14209 mini_oidc.cc:152] Starting JWKS server
I20260806 13:06:54.434680 14209 webserver.cc:460] Webserver started at https://127.0.0.1:42067/ using document root <none> and password file <none>
I20260806 13:06:54.434722 14209 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 13:06:54.434886 14209 webserver.cc:460] Webserver started at http://127.0.0.1:44283/ using document root <none> and password file <none>
2026-08-06T13:06:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:54Z Disabled control of system clock
I20260806 13:06:54.443264 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35399
--webserver_interface=127.13.224.126
--webserver_port=0
--builtin_ntp_servers=127.13.224.84:35739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:35399
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42067/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/testchainca.cert with env {}
W20260806 13:06:54.531092 21765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:54.531291 21765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:54.531312 21765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:54.532399 21765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:54.532431 21765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:54.532444 21765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:54.532454 21765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:54.532465 21765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 13:06:54.532478 21765 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 13:06:54.532491 21765 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42067/jwks.json
I20260806 13:06:54.533917 21765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:35739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:35399
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42067/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:54.534139 21765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:54.534309 21765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:54.538834 21770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:54.538854 21771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:54.538923 21765 server_base.cc:1034] running on GCE node
W20260806 13:06:54.538854 21774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:54.539269 21765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:54.539535 21765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:54.540683 21765 hybrid_clock.cc:648] HybridClock initialized: now 1786021614540670 us; error 28 us; skew 500 ppm
I20260806 13:06:54.545855 21765 webserver.cc:460] Webserver started at http://127.13.224.126:40967/ using document root <none> and password file <none>
I20260806 13:06:54.546092 21765 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:54.546144 21765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:54.546272 21765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:54.547339 21765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "3249f2d054624dd1ac32aca488abb223"
format_stamp: "Formatted at 2026-08-06 13:06:54 on dist-test-slave-q9h9"
server_key: "28928b51dc92e88ae8b84abdd496612d"
server_key_iv: "9967012b0a5a74157488130f0ee7763d"
server_key_version: "encryptionkey@0"
I20260806 13:06:54.547712 21765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "3249f2d054624dd1ac32aca488abb223"
format_stamp: "Formatted at 2026-08-06 13:06:54 on dist-test-slave-q9h9"
server_key: "28928b51dc92e88ae8b84abdd496612d"
server_key_iv: "9967012b0a5a74157488130f0ee7763d"
server_key_version: "encryptionkey@0"
I20260806 13:06:54.548908 21765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:06:54.549551 21785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:54.549708 21765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:54.549801 21765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "3249f2d054624dd1ac32aca488abb223"
format_stamp: "Formatted at 2026-08-06 13:06:54 on dist-test-slave-q9h9"
server_key: "28928b51dc92e88ae8b84abdd496612d"
server_key_iv: "9967012b0a5a74157488130f0ee7763d"
server_key_version: "encryptionkey@0"
I20260806 13:06:54.549862 21765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:54.563009 21765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:54.563269 21765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:54.563392 21765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:54.566977 21765 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:35399
I20260806 13:06:54.567013 21864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:35399 every 8 connection(s)
I20260806 13:06:54.567314 21765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:54.567945 21866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:54.570586 21866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223: Bootstrap starting.
I20260806 13:06:54.571105 21866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:54.571326 21866 log.cc:824] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:54.571918 21866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223: No bootstrap required, opened a new log
I20260806 13:06:54.573377 21866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3249f2d054624dd1ac32aca488abb223" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35399 } }
I20260806 13:06:54.573504 21866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:54.573531 21866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3249f2d054624dd1ac32aca488abb223, State: Initialized, Role: FOLLOWER
I20260806 13:06:54.573632 21866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3249f2d054624dd1ac32aca488abb223" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35399 } }
I20260806 13:06:54.573709 21866 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:54.573743 21866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:54.573776 21866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:54.574285 21866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3249f2d054624dd1ac32aca488abb223" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35399 } }
I20260806 13:06:54.574368 21866 leader_election.cc:302] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3249f2d054624dd1ac32aca488abb223; no voters:
I20260806 13:06:54.574491 21866 leader_election.cc:288] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:54.574535 21869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:54.574677 21869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [term 1 LEADER]: Becoming Leader. State: Replica: 3249f2d054624dd1ac32aca488abb223, State: Running, Role: LEADER
I20260806 13:06:54.574702 21866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:54.574779 21869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3249f2d054624dd1ac32aca488abb223" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35399 } }
I20260806 13:06:54.575141 21870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3249f2d054624dd1ac32aca488abb223" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3249f2d054624dd1ac32aca488abb223" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35399 } } }
I20260806 13:06:54.575254 21870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:54.575333 21871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3249f2d054624dd1ac32aca488abb223. Latest consensus state: current_term: 1 leader_uuid: "3249f2d054624dd1ac32aca488abb223" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3249f2d054624dd1ac32aca488abb223" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35399 } } }
I20260806 13:06:54.575420 21871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:54.575515 21883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:54.576009 21883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:54.577373 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 21765
I20260806 13:06:54.577482 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:54.577589 21883 catalog_manager.cc:1347] Generated new cluster ID: caed00c744044187bb181dbcc84b8ce4
I20260806 13:06:54.577646 21883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:06:54.577678 14209 external_mini_cluster.cc:1428] Setting key 02b8a17bf6b8c2a0c2926097febc4b07
I20260806 13:06:54.582267 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:54.578111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41158 (local address 127.13.224.126:35399)
0806 13:06:54.578380 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:54.578386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:54.578408 (+ 22us) server_negotiation.cc:407] Connection header received
0806 13:06:54.578499 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:54.578505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:54.578552 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:54.578658 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:54.578855 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.579732 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.580477 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.580895 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.581001 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:54.581584 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:54.581588 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:54.581588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:54.581765 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:54.581797 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:54.581807 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:54.581861 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:54.581935 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 13:06:54.582114 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":65,"threads_started":1}
I20260806 13:06:54.583277 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:06:54.594587 21883 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:54.595238 21883 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:54.599220 21883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3249f2d054624dd1ac32aca488abb223: Generated new TSK 0
I20260806 13:06:54.599370 21883 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 13:06:54.626148 21911 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 13:06:54.626207 21911 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 13:06:54.626222 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:54.604354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41164 (local address 127.13.224.126:35399)
0806 13:06:54.604396 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:54.604398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:54.604496 (+ 98us) server_negotiation.cc:407] Connection header received
0806 13:06:54.604567 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:54.604568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:54.604577 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:54.604633 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0806 13:06:54.604779 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.605254 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.606375 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.606566 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.606611 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:54.606770 (+ 159us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 13:06:54.625983 (+ 19213us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42067/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 13:06:54.626154 (+ 171us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41164: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42067/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 13:06:54.626266 21899 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41164: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42067/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 13:06:54.626574 21908 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:06:54.604298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35399 (local address 127.0.0.1:41164)
0806 13:06:54.604435 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:54.604446 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:54.604506 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:54.604651 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:54.604652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:54.604654 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 13:06:54.604714 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:54.604716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.605324 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.605325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:54.606308 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:54.606311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.606631 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.606632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:54.606690 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:54.606694 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 13:06:54.626363 (+ 19669us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42067/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 13:06:54.626479 (+ 116us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.224.126:35399: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42067/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260806 13:06:54.626942 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 21765
2026-08-06T13:06:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (237 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 13:06:54.671608 14209 mini_oidc.cc:152] Starting JWKS server
I20260806 13:06:54.672245 14209 webserver.cc:460] Webserver started at https://127.0.0.1:43435/ using document root <none> and password file <none>
I20260806 13:06:54.672283 14209 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 13:06:54.672397 14209 webserver.cc:460] Webserver started at http://127.0.0.1:37661/ using document root <none> and password file <none>
2026-08-06T13:06:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:54Z Disabled control of system clock
I20260806 13:06:54.680560 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:46007
--webserver_interface=127.13.224.126
--webserver_port=0
--builtin_ntp_servers=127.13.224.84:35073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:46007
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43435/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 13:06:54.759174 21921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:54.759361 21921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:54.759382 21921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:54.760474 21921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:54.760509 21921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:54.760520 21921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:54.760531 21921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:54.760546 21921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 13:06:54.760556 21921 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 13:06:54.760569 21921 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43435/jwks.json
I20260806 13:06:54.761855 21921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:35073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:46007
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:46007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43435/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:54.762084 21921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:54.762324 21921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:54.766822 21927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:54.766907 21921 server_base.cc:1034] running on GCE node
W20260806 13:06:54.766790 21933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:54.766796 21928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:54.767239 21921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:54.767513 21921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:54.768652 21921 hybrid_clock.cc:648] HybridClock initialized: now 1786021614768639 us; error 28 us; skew 500 ppm
I20260806 13:06:54.769829 21921 webserver.cc:460] Webserver started at http://127.13.224.126:35997/ using document root <none> and password file <none>
I20260806 13:06:54.770108 21921 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:54.770161 21921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:54.770290 21921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:54.771274 21921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "8a510424fb2d4e43892bfc32ea005c0a"
format_stamp: "Formatted at 2026-08-06 13:06:54 on dist-test-slave-q9h9"
server_key: "563e8089b14e311a2d2a390806127158"
server_key_iv: "2becab5fe678a10533c7c2cae480a6f4"
server_key_version: "encryptionkey@0"
I20260806 13:06:54.771590 21921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "8a510424fb2d4e43892bfc32ea005c0a"
format_stamp: "Formatted at 2026-08-06 13:06:54 on dist-test-slave-q9h9"
server_key: "563e8089b14e311a2d2a390806127158"
server_key_iv: "2becab5fe678a10533c7c2cae480a6f4"
server_key_version: "encryptionkey@0"
I20260806 13:06:54.772760 21921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:06:54.773406 21944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:54.773550 21921 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:54.773633 21921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "8a510424fb2d4e43892bfc32ea005c0a"
format_stamp: "Formatted at 2026-08-06 13:06:54 on dist-test-slave-q9h9"
server_key: "563e8089b14e311a2d2a390806127158"
server_key_iv: "2becab5fe678a10533c7c2cae480a6f4"
server_key_version: "encryptionkey@0"
I20260806 13:06:54.773698 21921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:54.792690 21921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:54.792982 21921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:54.793107 21921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:54.796792 21921 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:46007
I20260806 13:06:54.796844 22026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:46007 every 8 connection(s)
I20260806 13:06:54.797142 21921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:54.797807 22027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:54.800441 22027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a: Bootstrap starting.
I20260806 13:06:54.800967 22027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:54.801193 22027 log.cc:824] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:54.801807 22027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a: No bootstrap required, opened a new log
I20260806 13:06:54.803303 22027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a510424fb2d4e43892bfc32ea005c0a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46007 } }
I20260806 13:06:54.803436 22027 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:54.803457 22027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a510424fb2d4e43892bfc32ea005c0a, State: Initialized, Role: FOLLOWER
I20260806 13:06:54.803565 22027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a510424fb2d4e43892bfc32ea005c0a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46007 } }
I20260806 13:06:54.803638 22027 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:54.803673 22027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:54.803705 22027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:54.804198 22027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a510424fb2d4e43892bfc32ea005c0a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46007 } }
I20260806 13:06:54.804306 22027 leader_election.cc:302] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a510424fb2d4e43892bfc32ea005c0a; no voters:
I20260806 13:06:54.804315 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 21921
I20260806 13:06:54.804411 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:54.804473 22027 leader_election.cc:288] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:54.804523 22031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:54.804622 14209 external_mini_cluster.cc:1428] Setting key 7c14aaa39b641b30070013222c385b72
I20260806 13:06:54.804682 22031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [term 1 LEADER]: Becoming Leader. State: Replica: 8a510424fb2d4e43892bfc32ea005c0a, State: Running, Role: LEADER
I20260806 13:06:54.804765 22027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:54.804793 22031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a510424fb2d4e43892bfc32ea005c0a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46007 } }
I20260806 13:06:54.805115 22031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a510424fb2d4e43892bfc32ea005c0a. Latest consensus state: current_term: 1 leader_uuid: "8a510424fb2d4e43892bfc32ea005c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a510424fb2d4e43892bfc32ea005c0a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46007 } } }
I20260806 13:06:54.805197 22031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:54.805120 22032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a510424fb2d4e43892bfc32ea005c0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a510424fb2d4e43892bfc32ea005c0a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46007 } } }
I20260806 13:06:54.805256 22032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:54.806146 22041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:54.806663 22041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:54.807986 22041 catalog_manager.cc:1347] Generated new cluster ID: dc665520440342fe8d570ec8ce5ebc97
I20260806 13:06:54.808036 22041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:06:54.810807 22039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:54.805188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44310 (local address 127.13.224.126:46007)
0806 13:06:54.805582 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:54.805591 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:54.805624 (+ 33us) server_negotiation.cc:407] Connection header received
0806 13:06:54.805743 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:54.805751 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:54.805821 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:54.805968 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:54.806196 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.807637 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.808720 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.809328 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.809452 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:54.810112 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:54.810117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:54.810118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:54.810338 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:54.810380 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:54.810394 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:54.810465 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:54.810521 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 13:06:54.810602 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":249,"threads_started":1}
I20260806 13:06:54.811632 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:06:54.822847 22041 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:54.823534 22041 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:54.830624 22041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a510424fb2d4e43892bfc32ea005c0a: Generated new TSK 0
I20260806 13:06:54.830783 22041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:06:54.844954 22039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:54.841530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44324 (local address 127.13.224.126:46007)
0806 13:06:54.841582 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:54.841584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:54.841683 (+ 99us) server_negotiation.cc:407] Connection header received
0806 13:06:54.841764 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:54.841765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:54.841775 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:54.841829 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0806 13:06:54.841948 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.842460 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.843554 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.843756 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.843807 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:54.843965 (+ 158us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 13:06:54.844758 (+ 793us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 13:06:54.844882 (+ 124us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44324: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":29}
W20260806 13:06:54.845009 22039 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44324: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 13:06:54.845124 22074 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:06:54.841468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:46007 (local address 127.0.0.1:44324)
0806 13:06:54.841618 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:54.841629 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:54.841694 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:54.841842 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:54.841842 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:54.841845 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0806 13:06:54.841897 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:54.841899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.842530 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.842531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:54.843476 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:54.843479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:54.843824 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:54.843825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:54.843883 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:54.843888 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 13:06:54.844855 (+ 967us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 13:06:54.844954 (+ 99us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.224.126:46007: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43435/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260806 13:06:54.845353 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 21921
E20260806 13:06:54.870110 22075 webserver.cc:533] Webserver: sslize failed:
E20260806 13:06:54.870177 22075 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T13:06:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (225 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T13:06:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:54Z Disabled control of system clock
I20260806 13:06:54.904762 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:45753
--webserver_interface=127.13.224.126
--webserver_port=0
--builtin_ntp_servers=127.13.224.84:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:45753
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/testchainca.cert with env {}
W20260806 13:06:54.985507 22081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:54.985709 22081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:54.985740 22081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:54.986901 22081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:54.986948 22081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:54.986968 22081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:54.986987 22081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:54.987007 22081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 13:06:54.987032 22081 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 13:06:54.987067 22081 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 13:06:54.988591 22081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:45753
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:45753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:54.988849 22081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:54.989064 22081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:54.993592 22089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:54.993607 22092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:54.993602 22088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:54.993826 22081 server_base.cc:1034] running on GCE node
I20260806 13:06:54.994076 22081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:54.994346 22081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:54.995472 22081 hybrid_clock.cc:648] HybridClock initialized: now 1786021614995460 us; error 26 us; skew 500 ppm
I20260806 13:06:54.996587 22081 webserver.cc:460] Webserver started at http://127.13.224.126:43593/ using document root <none> and password file <none>
I20260806 13:06:54.996815 22081 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:54.996863 22081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:54.996987 22081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:54.998009 22081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "21c734b6d3a54416af5e0202b12b1225"
format_stamp: "Formatted at 2026-08-06 13:06:54 on dist-test-slave-q9h9"
server_key: "c655b6a37334cf91cf7dd42b621d0fa8"
server_key_iv: "c0d5f20b848f24c63e56ad62217e32ab"
server_key_version: "encryptionkey@0"
I20260806 13:06:54.998435 22081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "21c734b6d3a54416af5e0202b12b1225"
format_stamp: "Formatted at 2026-08-06 13:06:54 on dist-test-slave-q9h9"
server_key: "c655b6a37334cf91cf7dd42b621d0fa8"
server_key_iv: "c0d5f20b848f24c63e56ad62217e32ab"
server_key_version: "encryptionkey@0"
I20260806 13:06:54.999701 22081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:06:55.000331 22101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:55.000483 22081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:55.000561 22081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "21c734b6d3a54416af5e0202b12b1225"
format_stamp: "Formatted at 2026-08-06 13:06:54 on dist-test-slave-q9h9"
server_key: "c655b6a37334cf91cf7dd42b621d0fa8"
server_key_iv: "c0d5f20b848f24c63e56ad62217e32ab"
server_key_version: "encryptionkey@0"
I20260806 13:06:55.000622 22081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:55.019212 22081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:55.019486 22081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:55.019606 22081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:55.023299 22081 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:45753
I20260806 13:06:55.023334 22184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:45753 every 8 connection(s)
I20260806 13:06:55.023643 22081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:55.024602 22185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:55.026871 22185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225: Bootstrap starting.
I20260806 13:06:55.027415 22185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:55.027652 22185 log.cc:824] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:55.028249 22185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225: No bootstrap required, opened a new log
I20260806 13:06:55.028647 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 22081
I20260806 13:06:55.028740 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:55.028891 14209 external_mini_cluster.cc:1428] Setting key ec7f9c89591ee5bbe557fe0148372582
I20260806 13:06:55.030108 22185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21c734b6d3a54416af5e0202b12b1225" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45753 } }
I20260806 13:06:55.030277 22185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:55.030316 22185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c734b6d3a54416af5e0202b12b1225, State: Initialized, Role: FOLLOWER
I20260806 13:06:55.030443 22185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21c734b6d3a54416af5e0202b12b1225" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45753 } }
I20260806 13:06:55.030529 22185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:55.030550 22185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:55.030596 22185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:55.031118 22185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21c734b6d3a54416af5e0202b12b1225" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45753 } }
I20260806 13:06:55.031208 22185 leader_election.cc:302] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21c734b6d3a54416af5e0202b12b1225; no voters:
I20260806 13:06:55.031373 22185 leader_election.cc:288] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:55.031404 22197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:55.031514 22197 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [term 1 LEADER]: Becoming Leader. State: Replica: 21c734b6d3a54416af5e0202b12b1225, State: Running, Role: LEADER
I20260806 13:06:55.031618 22185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:55.031594 22197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21c734b6d3a54416af5e0202b12b1225" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45753 } }
I20260806 13:06:55.032028 22199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21c734b6d3a54416af5e0202b12b1225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21c734b6d3a54416af5e0202b12b1225" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45753 } } }
I20260806 13:06:55.032037 22198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21c734b6d3a54416af5e0202b12b1225. Latest consensus state: current_term: 1 leader_uuid: "21c734b6d3a54416af5e0202b12b1225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21c734b6d3a54416af5e0202b12b1225" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45753 } } }
I20260806 13:06:55.032187 22199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:55.032207 22198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:55.032980 22210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:55.033458 22210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:55.035041 22210 catalog_manager.cc:1347] Generated new cluster ID: 193ab8a242524a07911ef749ee35c63b
I20260806 13:06:55.035095 22210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:06:55.038300 22194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:55.029340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.13.224.126:45753)
0806 13:06:55.029523 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:55.029531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:55.029553 (+ 22us) server_negotiation.cc:407] Connection header received
0806 13:06:55.030053 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:55.030059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:55.030113 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:55.030233 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:55.030447 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:55.032020 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:55.033087 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:55.033697 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:55.034118 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:55.034551 (+ 433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:55.034555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:55.034555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:06:55.034767 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:55.034820 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:06:55.034835 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:55.034906 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:55.038068 (+ 3162us) server_negotiation.cc:299] Negotiation successful
0806 13:06:55.038141 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260806 13:06:55.039080 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:06:55.061517 22210 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:55.061996 22210 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:55.070303 22210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21c734b6d3a54416af5e0202b12b1225: Generated new TSK 0
I20260806 13:06:55.070458 22210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:06:55.626332 22194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:55.075380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46116 (local address 127.13.224.126:45753)
0806 13:06:55.075439 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:55.075442 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:55.075531 (+ 89us) server_negotiation.cc:407] Connection header received
0806 13:06:55.075612 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:55.075612 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:55.075622 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:55.075678 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0806 13:06:55.075808 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:55.076271 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:55.077379 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:55.077560 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:55.077618 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:55.077783 (+ 165us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 13:06:55.625991 (+548208us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 13:06:55.626233 (+ 242us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46116: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 13:06:55.626417 22194 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46116: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 13:06:55.626451 22231 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:06:55.075325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:45753 (local address 127.0.0.1:46116)
0806 13:06:55.075475 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:55.075486 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:55.075549 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:55.075688 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:55.075689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:55.075691 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 13:06:55.075751 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:55.075753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:55.076332 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:55.076333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:55.077283 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:55.077286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:55.077636 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:55.077637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:55.077698 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:55.077703 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 13:06:55.626196 (+548493us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 13:06:55.626349 (+ 153us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.224.126:45753: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260806 13:06:55.626932 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 22081
2026-08-06T13:06:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (754 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:06:55 dist-test-slave-q9h9 krb5kdc[22258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:06:55 dist-test-slave-q9h9 krb5kdc[22258](info): set up 2 sockets
Aug 06 13:06:55 dist-test-slave-q9h9 krb5kdc[22258](info): commencing operation
krb5kdc: starting...
W20260806 13:06:56.925906 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:06:56 dist-test-slave-q9h9 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:06:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:06:57.033803 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:33039
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:37165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:33039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:57.114171 22281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:57.114353 22281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:57.114373 22281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:57.115530 22281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:57.115571 22281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:57.115583 22281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:57.115594 22281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:57.115608 22281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:06:57.117067 22281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:37165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:33039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:33039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:57.117295 22281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:57.117517 22281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:57.122053 22289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:57.122062 22293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:57.122215 22290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:57.122216 22281 server_base.cc:1034] running on GCE node
I20260806 13:06:57.122546 22281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:57.122802 22281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:57.123973 22281 hybrid_clock.cc:648] HybridClock initialized: now 1786021617123954 us; error 38 us; skew 500 ppm
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:57.126519 22281 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:06:57.126902 22281 webserver.cc:460] Webserver started at http://127.13.224.126:40201/ using document root <none> and password file <none>
I20260806 13:06:57.127104 22281 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:57.127161 22281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:57.127254 22281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:57.128219 22281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "b8900d8b494e4d47a3904046ff020032"
format_stamp: "Formatted at 2026-08-06 13:06:57 on dist-test-slave-q9h9"
server_key: "a822297804347521a3c520abba3c91f7"
server_key_iv: "743f806ae4a3184ef6ef3071b07fbcdd"
server_key_version: "encryptionkey@0"
I20260806 13:06:57.128515 22281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "b8900d8b494e4d47a3904046ff020032"
format_stamp: "Formatted at 2026-08-06 13:06:57 on dist-test-slave-q9h9"
server_key: "a822297804347521a3c520abba3c91f7"
server_key_iv: "743f806ae4a3184ef6ef3071b07fbcdd"
server_key_version: "encryptionkey@0"
I20260806 13:06:57.129758 22281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:06:57.130461 22304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:57.130618 22281 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:57.130712 22281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "b8900d8b494e4d47a3904046ff020032"
format_stamp: "Formatted at 2026-08-06 13:06:57 on dist-test-slave-q9h9"
server_key: "a822297804347521a3c520abba3c91f7"
server_key_iv: "743f806ae4a3184ef6ef3071b07fbcdd"
server_key_version: "encryptionkey@0"
I20260806 13:06:57.130785 22281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:57.145907 22281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:57.147680 22281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:57.147804 22281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:57.151511 22281 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:33039
I20260806 13:06:57.151559 22391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:33039 every 8 connection(s)
I20260806 13:06:57.151866 22281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:57.152698 22392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:57.154978 22392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032: Bootstrap starting.
I20260806 13:06:57.155520 22392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:57.155794 22392 log.cc:824] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:57.156410 22392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032: No bootstrap required, opened a new log
I20260806 13:06:57.158057 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 22281
I20260806 13:06:57.157944 22392 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8900d8b494e4d47a3904046ff020032" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33039 } }
I20260806 13:06:57.158111 22392 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:57.158149 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:57.158149 22392 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8900d8b494e4d47a3904046ff020032, State: Initialized, Role: FOLLOWER
I20260806 13:06:57.158280 22392 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8900d8b494e4d47a3904046ff020032" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33039 } }
I20260806 13:06:57.158335 14209 external_mini_cluster.cc:1428] Setting key 820803522e1e5f0b89ef0a819016bbdd
I20260806 13:06:57.158357 22392 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:57.158391 22392 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:57.158440 22392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:57.159037 22392 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8900d8b494e4d47a3904046ff020032" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33039 } }
I20260806 13:06:57.159125 22392 leader_election.cc:302] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8900d8b494e4d47a3904046ff020032; no voters:
I20260806 13:06:57.159276 22392 leader_election.cc:288] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:57.159322 22396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:57.159475 22396 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [term 1 LEADER]: Becoming Leader. State: Replica: b8900d8b494e4d47a3904046ff020032, State: Running, Role: LEADER
I20260806 13:06:57.159531 22392 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:57.159559 22396 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8900d8b494e4d47a3904046ff020032" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33039 } }
I20260806 13:06:57.159904 22396 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8900d8b494e4d47a3904046ff020032. Latest consensus state: current_term: 1 leader_uuid: "b8900d8b494e4d47a3904046ff020032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8900d8b494e4d47a3904046ff020032" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33039 } } }
I20260806 13:06:57.159969 22396 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:57.159933 22398 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8900d8b494e4d47a3904046ff020032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8900d8b494e4d47a3904046ff020032" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33039 } } }
I20260806 13:06:57.160012 22398 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032 [sys.catalog]: This master's current role is: LEADER
W20260806 13:06:57.161352 22420 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:06:57.161407 22420 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:06:57.161451 22405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:57.161598 22405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:57.163107 22405 catalog_manager.cc:1347] Generated new cluster ID: 6f660abcc7cf48e4ae2ce30c301124a0
I20260806 13:06:57.163160 22405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:57.168500 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:57.158752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33148 (local address 127.13.224.126:33039)
0806 13:06:57.159441 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:57.159451 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:57.159482 (+ 31us) server_negotiation.cc:407] Connection header received
0806 13:06:57.159599 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:57.159606 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:57.159660 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:57.159789 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:57.160594 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.161724 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.162693 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.163224 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.163323 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:57.165467 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:57.165471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:57.165472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:57.165483 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:57.167120 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.167435 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.167438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.167438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.167538 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.167708 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.167709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.167709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.167946 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:57.168101 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:57.168168 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 13:06:57.168312 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":576,"threads_started":1}
I20260806 13:06:57.177270 22405 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:57.177735 22405 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:57.182458 22405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8900d8b494e4d47a3904046ff020032: Generated new TSK 0
I20260806 13:06:57.182621 22405 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:06:57.212522 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33039
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:37165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:57.293042 22428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:57.293246 22428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:57.293277 22428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:57.294600 22428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:57.294654 22428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:57.294685 22428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:06:57.296334 22428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:37165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:57.296510 22428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:57.296713 22428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:57.301369 22437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:57.301412 22442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:57.301352 22436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:57.301535 22428 server_base.cc:1034] running on GCE node
I20260806 13:06:57.301774 22428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:57.301975 22428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:57.303115 22428 hybrid_clock.cc:648] HybridClock initialized: now 1786021617303106 us; error 26 us; skew 500 ppm
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:57.305374 22428 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:06:57.305768 22428 webserver.cc:460] Webserver started at http://127.13.224.65:37305/ using document root <none> and password file <none>
I20260806 13:06:57.305955 22428 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:57.306001 22428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:57.306149 22428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:57.307106 22428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "ada95b96366a4be6b7a1d935345501d3"
format_stamp: "Formatted at 2026-08-06 13:06:57 on dist-test-slave-q9h9"
server_key: "7c1c6477fdc0fc91e140aedb561a2a5b"
server_key_iv: "c5a198d6ed7925d1ac9d30bfa4debf9c"
server_key_version: "encryptionkey@0"
I20260806 13:06:57.307421 22428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "ada95b96366a4be6b7a1d935345501d3"
format_stamp: "Formatted at 2026-08-06 13:06:57 on dist-test-slave-q9h9"
server_key: "7c1c6477fdc0fc91e140aedb561a2a5b"
server_key_iv: "c5a198d6ed7925d1ac9d30bfa4debf9c"
server_key_version: "encryptionkey@0"
I20260806 13:06:57.308720 22428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:06:57.309355 22453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:57.309499 22428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:57.309580 22428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "ada95b96366a4be6b7a1d935345501d3"
format_stamp: "Formatted at 2026-08-06 13:06:57 on dist-test-slave-q9h9"
server_key: "7c1c6477fdc0fc91e140aedb561a2a5b"
server_key_iv: "c5a198d6ed7925d1ac9d30bfa4debf9c"
server_key_version: "encryptionkey@0"
I20260806 13:06:57.309638 22428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:57.335613 22428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:57.337163 22428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:57.337296 22428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:57.337498 22428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:57.337795 22428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:57.337829 22428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:57.337855 22428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:57.337872 22428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:57.343379 22428 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:36725
I20260806 13:06:57.343438 22612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:36725 every 8 connection(s)
I20260806 13:06:57.343773 22428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:06:57.347223 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 22428
I20260806 13:06:57.347344 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:06:57.347538 14209 external_mini_cluster.cc:1428] Setting key 56364e5dd7ead6bbcb6a84f17c300071
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:57.353030 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:57.344418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:47329 (local address 127.13.224.126:33039)
0806 13:06:57.344471 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:57.344473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:57.344639 (+ 166us) server_negotiation.cc:407] Connection header received
0806 13:06:57.345509 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:57.345511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:57.345527 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:57.345596 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:57.346213 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.346760 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.347979 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.348220 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.348278 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:57.350473 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:57.350475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:57.350475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:57.350483 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:57.351716 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.352141 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.352141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.352142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.352190 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.352453 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.352454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.352454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.352544 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:57.352598 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:57.352857 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 13:06:57.352940 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 13:06:57.353160 22617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:57.344391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33039 (local address 127.13.224.65:47329)
0806 13:06:57.344531 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:57.344566 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:57.345408 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:57.345663 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:57.345668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:57.345919 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:57.346119 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:57.346124 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.346841 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.346842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:57.347907 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:57.347909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.348294 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.348295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:57.348431 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:57.348879 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:57.348889 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:57.350383 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:57.351795 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:57.351803 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:57.351806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:57.352073 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:57.352250 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:57.352251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:57.352251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:57.352394 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:57.352609 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:57.352617 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:57.352727 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 13:06:57.352822 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 13:06:57.352964 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
I20260806 13:06:57.353614 22614 heartbeater.cc:347] Connected to a master server at 127.13.224.126:33039
I20260806 13:06:57.353726 22614 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:57.353926 22614 heartbeater.cc:511] Master 127.13.224.126:33039 requested a full tablet report, sending...
I20260806 13:06:57.354489 22335 ts_manager.cc:194] Registered new tserver with Master: ada95b96366a4be6b7a1d935345501d3 (127.13.224.65:36725)
I20260806 13:06:57.355381 22335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:47329
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:06:57.370718 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33039
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:37165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:57.449988 22623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:57.450198 22623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:57.450222 22623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:57.451311 22623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:57.451345 22623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:57.451370 22623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:06:57.452795 22623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:37165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:57.452972 22623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:57.453154 22623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:57.457965 22635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:57.457918 22638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:57.457926 22634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:57.457978 22623 server_base.cc:1034] running on GCE node
I20260806 13:06:57.458353 22623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:57.458556 22623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:57.459693 22623 hybrid_clock.cc:648] HybridClock initialized: now 1786021617459683 us; error 27 us; skew 500 ppm
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:57.462149 22623 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:06:57.462527 22623 webserver.cc:460] Webserver started at http://127.13.224.66:44223/ using document root <none> and password file <none>
I20260806 13:06:57.462711 22623 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:57.462752 22623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:57.462843 22623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:57.463792 22623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "9950142881434828abaa781620424406"
format_stamp: "Formatted at 2026-08-06 13:06:57 on dist-test-slave-q9h9"
server_key: "bc4d742f08923a40524a27c55ac275c3"
server_key_iv: "47e7a177cc87215bad23f7d874d23291"
server_key_version: "encryptionkey@0"
I20260806 13:06:57.464079 22623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "9950142881434828abaa781620424406"
format_stamp: "Formatted at 2026-08-06 13:06:57 on dist-test-slave-q9h9"
server_key: "bc4d742f08923a40524a27c55ac275c3"
server_key_iv: "47e7a177cc87215bad23f7d874d23291"
server_key_version: "encryptionkey@0"
I20260806 13:06:57.465250 22623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:06:57.465879 22648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:57.466042 22623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:57.466126 22623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "9950142881434828abaa781620424406"
format_stamp: "Formatted at 2026-08-06 13:06:57 on dist-test-slave-q9h9"
server_key: "bc4d742f08923a40524a27c55ac275c3"
server_key_iv: "47e7a177cc87215bad23f7d874d23291"
server_key_version: "encryptionkey@0"
I20260806 13:06:57.466197 22623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:57.489508 22623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:57.491626 22623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:57.491770 22623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:57.492003 22623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:57.492318 22623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:57.492352 22623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:57.492374 22623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:57.492398 22623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:57.497925 22623 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:35049
I20260806 13:06:57.497972 22816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:35049 every 8 connection(s)
I20260806 13:06:57.498332 22623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:57.504976 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 22623
I20260806 13:06:57.505076 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:06:57.505287 14209 external_mini_cluster.cc:1428] Setting key 96675e0522b8106a78600def70e85fe9
I20260806 13:06:57.507120 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:57.499035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:39377 (local address 127.13.224.126:33039)
0806 13:06:57.499102 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:57.499104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:57.499288 (+ 184us) server_negotiation.cc:407] Connection header received
0806 13:06:57.500169 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:57.500170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:57.500180 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:57.500244 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:57.500806 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.501295 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.502260 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.502438 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.502483 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:57.504435 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:57.504437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:57.504437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:57.504446 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:57.505845 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.506302 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.506303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.506303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.506351 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.506585 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.506586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.506586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.506677 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:57.506721 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:57.506999 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 13:06:57.507056 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 13:06:57.507340 22825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:57.498988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33039 (local address 127.13.224.66:39377)
0806 13:06:57.499158 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:57.499201 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:57.500088 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:57.500310 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:57.500314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:57.500553 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:57.500740 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:57.500745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.501355 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.501356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:57.502177 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:57.502180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.502490 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.502491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:57.502584 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:57.502974 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:57.502981 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:57.504342 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:57.505920 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:57.505928 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:57.505930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:57.506240 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:57.506405 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:57.506405 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:57.506406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:57.506533 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:57.506725 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:57.506729 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:57.506885 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 13:06:57.506980 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 13:06:57.507130 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260806 13:06:57.508064 22820 heartbeater.cc:347] Connected to a master server at 127.13.224.126:33039
I20260806 13:06:57.508170 22820 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:57.508359 22820 heartbeater.cc:511] Master 127.13.224.126:33039 requested a full tablet report, sending...
I20260806 13:06:57.508723 22335 ts_manager.cc:194] Registered new tserver with Master: 9950142881434828abaa781620424406 (127.13.224.66:35049)
I20260806 13:06:57.509148 22335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:39377
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:06:57.528682 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33039
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:37165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:57.608237 22829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:57.608439 22829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:57.608472 22829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:57.609606 22829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:57.609645 22829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:57.609679 22829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:06:57.611156 22829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:37165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:57.611402 22829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:57.611601 22829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:57.616459 22835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:57.616621 22829 server_base.cc:1034] running on GCE node
W20260806 13:06:57.616573 22839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:57.616466 22837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:57.616931 22829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:57.617164 22829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:57.618294 22829 hybrid_clock.cc:648] HybridClock initialized: now 1786021617618283 us; error 28 us; skew 500 ppm
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:57.620585 22829 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:06:57.620980 22829 webserver.cc:460] Webserver started at http://127.13.224.67:38421/ using document root <none> and password file <none>
I20260806 13:06:57.621165 22829 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:57.621204 22829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:57.621304 22829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:57.622304 22829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "5cf1daa190674b73a150e698f74137a5"
format_stamp: "Formatted at 2026-08-06 13:06:57 on dist-test-slave-q9h9"
server_key: "7b6ea2c953a03674dd648b5721f71cef"
server_key_iv: "04bd7940de4886a99e73af8067973cbd"
server_key_version: "encryptionkey@0"
I20260806 13:06:57.622596 22829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "5cf1daa190674b73a150e698f74137a5"
format_stamp: "Formatted at 2026-08-06 13:06:57 on dist-test-slave-q9h9"
server_key: "7b6ea2c953a03674dd648b5721f71cef"
server_key_iv: "04bd7940de4886a99e73af8067973cbd"
server_key_version: "encryptionkey@0"
I20260806 13:06:57.623729 22829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:06:57.624361 22848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:57.624505 22829 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:57.624586 22829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "5cf1daa190674b73a150e698f74137a5"
format_stamp: "Formatted at 2026-08-06 13:06:57 on dist-test-slave-q9h9"
server_key: "7b6ea2c953a03674dd648b5721f71cef"
server_key_iv: "04bd7940de4886a99e73af8067973cbd"
server_key_version: "encryptionkey@0"
I20260806 13:06:57.624646 22829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:57.644357 22829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:57.646092 22829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:57.646261 22829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:57.646497 22829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:57.646798 22829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:57.646831 22829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:57.646860 22829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:57.646876 22829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:57.652592 22829 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:40089
I20260806 13:06:57.652652 23032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:40089 every 8 connection(s)
I20260806 13:06:57.652940 22829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:06:57.661933 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:57.653623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:43097 (local address 127.13.224.126:33039)
0806 13:06:57.653686 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:57.653688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:57.653888 (+ 200us) server_negotiation.cc:407] Connection header received
0806 13:06:57.654924 (+ 1036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:57.654925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:57.654935 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:57.654987 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:57.655557 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.656047 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.657010 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.657182 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.657223 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:57.659295 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:57.659297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:57.659298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:57.659306 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:57.660608 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.661064 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.661066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.661066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.661118 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.661341 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.661342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.661343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.661450 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:57.661506 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:57.661765 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 13:06:57.661843 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 13:06:57.662120 23038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:57.653595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33039 (local address 127.13.224.67:43097)
0806 13:06:57.653761 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:57.653804 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:57.654836 (+ 1032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:57.655049 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:57.655053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:57.655303 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:57.655493 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:57.655497 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.656113 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.656113 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:57.656947 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:57.656949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.657234 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.657235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:57.657328 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:57.657742 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:57.657750 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:57.659205 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:57.660686 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:57.660693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:57.660695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:57.660995 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:57.661173 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:57.661174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:57.661174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:57.661286 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:57.661508 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:57.661515 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:57.661642 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 13:06:57.661729 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 13:06:57.661880 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260806 13:06:57.662560 23034 heartbeater.cc:347] Connected to a master server at 127.13.224.126:33039
I20260806 13:06:57.662648 23034 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:57.662832 23034 heartbeater.cc:511] Master 127.13.224.126:33039 requested a full tablet report, sending...
I20260806 13:06:57.662899 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 22829
I20260806 13:06:57.662992 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:06:57.663177 14209 external_mini_cluster.cc:1428] Setting key 514488e3798a1c5ef74ea17d0bdd36c5
I20260806 13:06:57.663229 22335 ts_manager.cc:194] Registered new tserver with Master: 5cf1daa190674b73a150e698f74137a5 (127.13.224.67:40089)
I20260806 13:06:57.663605 22335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:43097
I20260806 13:06:57.663988 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/test-admin.keytab.
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:57.678162 14209 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 13:06:57.679610 23045 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:06:57.678966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33764 (local address 127.13.224.65:36725)
0806 13:06:57.679130 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:57.679135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:57.679144 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:06:57.679167 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:57.679169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:57.679182 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:57.679247 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:57.679547 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33764: BlockingRecv error: recv got EOF from 127.0.0.1:33764 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
W20260806 13:06:57.679817 14209 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/krb5cc' of type FILE
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:57.680671 14209 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:57.681419 14209 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.65@KRBTEST.COM
I20260806 13:06:57.688331 23045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:57.681869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33768 (local address 127.13.224.65:36725)
0806 13:06:57.681917 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:57.681918 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:57.681927 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:06:57.681983 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:57.681984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:57.681994 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:57.682076 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:57.682429 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.683198 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.683882 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.684188 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.684253 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:57.686069 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:57.686077 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:57.686078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:57.686084 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:57.687303 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.687599 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.687601 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.687602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.687655 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.687825 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.687826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.687826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.687950 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:57.688071 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:57.688157 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 13:06:57.688228 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260806 13:06:57.689823 23045 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:06:57.689407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33774 (local address 127.13.224.65:36725)
0806 13:06:57.689443 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:57.689445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:57.689479 (+ 34us) server_negotiation.cc:407] Connection header received
0806 13:06:57.689525 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:57.689526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:57.689534 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:57.689575 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:57.689785 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33774: BlockingRecv error: recv got EOF from 127.0.0.1:33774 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 13:06:57.689903 14209 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/krb5cc' of type FILE
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:57.690761 14209 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:57.691454 14209 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.65@KRBTEST.COM
I20260806 13:06:57.697229 23045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:57.691902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33776 (local address 127.13.224.65:36725)
0806 13:06:57.691949 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:57.691950 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:57.691999 (+ 49us) server_negotiation.cc:407] Connection header received
0806 13:06:57.692044 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:57.692045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:57.692053 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:57.692087 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:57.692373 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.692826 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.693405 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.693579 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.693620 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:57.695329 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:57.695330 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:57.695330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:57.695336 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:57.696504 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.696736 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.696737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.696737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.696782 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.696894 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.696894 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.696894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.696967 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:57.697024 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:57.697092 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 13:06:57.697148 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260806 13:06:57.698613 23045 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:06:57.698147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33782 (local address 127.13.224.65:36725)
0806 13:06:57.698192 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:57.698193 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:57.698291 (+ 98us) server_negotiation.cc:407] Connection header received
0806 13:06:57.698348 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:57.698348 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:57.698355 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:57.698396 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:57.698573 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33782: BlockingRecv error: recv got EOF from 127.0.0.1:33782 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 13:06:57.698760 14209 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786021591455662-14209-0/krb5kdc/krb5cc)
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:57.699563 14209 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:57.700259 14209 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.65@KRBTEST.COM
I20260806 13:06:57.705840 23045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:57.700685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33784 (local address 127.13.224.65:36725)
0806 13:06:57.700718 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:57.700719 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:57.700742 (+ 23us) server_negotiation.cc:407] Connection header received
0806 13:06:57.700790 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:57.700791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:57.700797 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:57.700836 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:57.701106 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.701550 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.702182 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:57.702369 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:57.702417 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:57.704026 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:57.704027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:57.704027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:57.704033 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:57.705140 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.705375 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.705376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.705376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.705417 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:57.705528 (+ 111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:57.705529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:57.705529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:57.705600 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:57.705656 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:57.705722 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 13:06:57.705769 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 13:06:57.706434 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 22428
I20260806 13:06:57.725097 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 22623
I20260806 13:06:57.740953 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 22829
I20260806 13:06:57.749563 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 22281
2026-08-06T13:06:57Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1946 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[23065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[23065](info): set up 2 sockets
Aug 06 13:06:57 dist-test-slave-q9h9 krb5kdc[23065](info): commencing operation
krb5kdc: starting...
W20260806 13:06:59.132701 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.358s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:06:59 dist-test-slave-q9h9 krb5kdc[23065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:06:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:06:59Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.126.keytab.
Entry for principal oryx/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.126.keytab.
I20260806 13:06:59.305202 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35805
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.126.keytab
--principal=oryx/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:35805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:59.442901 23087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:59.443254 23087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:59.443289 23087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:59.445076 23087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:06:59.445142 23087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:59.445160 23087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:59.445199 23087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:06:59.445241 23087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:06:59.447655 23087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:35805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.126.keytab
--principal=oryx/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:59.448006 23087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:59.448331 23087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:59.455595 23094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:59.455595 23098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:59.455703 23096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:59.456635 23087 server_base.cc:1034] running on GCE node
I20260806 13:06:59.457011 23087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:59.457434 23087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:59.458626 23087 hybrid_clock.cc:648] HybridClock initialized: now 1786021619458596 us; error 50 us; skew 500 ppm
Aug 06 13:06:59 dist-test-slave-q9h9 krb5kdc[23065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021619, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:59.462590 23087 init.cc:377] Logged in from keytab as oryx/127.13.224.126@KRBTEST.COM (short username oryx)
I20260806 13:06:59.463233 23087 webserver.cc:460] Webserver started at http://127.13.224.126:35703/ using document root <none> and password file <none>
I20260806 13:06:59.463546 23087 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:59.463649 23087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:59.463918 23087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:59.465473 23087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "bf238ca059c646dcaa34f2ee20cd1917"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "e29155be41d7a357930521dc009b5b81"
server_key_iv: "5e736aaccacd58d524405857fdeaa4e3"
server_key_version: "encryptionkey@0"
I20260806 13:06:59.466066 23087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "bf238ca059c646dcaa34f2ee20cd1917"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "e29155be41d7a357930521dc009b5b81"
server_key_iv: "5e736aaccacd58d524405857fdeaa4e3"
server_key_version: "encryptionkey@0"
I20260806 13:06:59.468395 23087 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260806 13:06:59.469621 23107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:59.469864 23087 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:06:59.470003 23087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "bf238ca059c646dcaa34f2ee20cd1917"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "e29155be41d7a357930521dc009b5b81"
server_key_iv: "5e736aaccacd58d524405857fdeaa4e3"
server_key_version: "encryptionkey@0"
I20260806 13:06:59.470158 23087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:59.491792 23087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:59.495101 23087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:59.495378 23087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:59.503054 23087 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:35805
I20260806 13:06:59.503109 23187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:35805 every 8 connection(s)
I20260806 13:06:59.503829 23087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:06:59.505708 23190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:06:59.509619 23190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917: Bootstrap starting.
I20260806 13:06:59.510710 23190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917: Neither blocks nor log segments found. Creating new log.
I20260806 13:06:59.511198 23190 log.cc:824] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917: Log is configured to *not* fsync() on all Append() calls
I20260806 13:06:59.512271 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 23087
I20260806 13:06:59.512444 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:06:59.512421 23190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917: No bootstrap required, opened a new log
I20260806 13:06:59.512704 14209 external_mini_cluster.cc:1428] Setting key c8bb7f946bfd897db92f0bf62ab171ab
I20260806 13:06:59.517071 23190 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } }
I20260806 13:06:59.517496 23190 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:06:59.517635 23190 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf238ca059c646dcaa34f2ee20cd1917, State: Initialized, Role: FOLLOWER
I20260806 13:06:59.518105 23190 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } }
I20260806 13:06:59.518314 23190 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:06:59.518392 23190 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:06:59.518446 23190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:06:59.519552 23190 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } }
I20260806 13:06:59.519731 23190 leader_election.cc:302] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf238ca059c646dcaa34f2ee20cd1917; no voters:
I20260806 13:06:59.519991 23190 leader_election.cc:288] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:06:59.520037 23198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:06:59.520352 23198 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [term 1 LEADER]: Becoming Leader. State: Replica: bf238ca059c646dcaa34f2ee20cd1917, State: Running, Role: LEADER
I20260806 13:06:59.520440 23190 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:06:59.520601 23198 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } }
I20260806 13:06:59.521217 23199 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } }
I20260806 13:06:59.521356 23199 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:59.521736 23200 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf238ca059c646dcaa34f2ee20cd1917. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } }
I20260806 13:06:59.521921 23200 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: This master's current role is: LEADER
I20260806 13:06:59.522502 23212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:06:59.523492 23212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:06:59.526331 23212 catalog_manager.cc:1347] Generated new cluster ID: 757a651f8e0b4b919e896eb6b1b83831
I20260806 13:06:59.526414 23212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:06:59 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
I20260806 13:06:59.534046 23212 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:06:59.534307 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:59.513481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46156 (local address 127.13.224.126:35805)
0806 13:06:59.513826 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:59.513847 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:59.513892 (+ 45us) server_negotiation.cc:407] Connection header received
0806 13:06:59.515192 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:59.515212 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:59.515365 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:59.515758 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:59.516971 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:59.520086 (+ 3115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:59.521728 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:59.522867 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:59.523023 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:59.527855 (+ 4832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:59.527859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:59.527860 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:59.527882 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:59.531215 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:59.531902 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:59.531915 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:59.531915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:59.532074 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:59.532389 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:59.532390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:59.532391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:59.532960 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:59.533232 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:59.533430 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 13:06:59.533870 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":202,"threads_started":1}
I20260806 13:06:59.534981 23212 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:06:59.543545 23212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917: Generated new TSK 0
I20260806 13:06:59.543808 23212 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab.
Entry for principal oryx/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab.
I20260806 13:06:59.578727 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35805
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab
--principal=oryx/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:59.660964 23234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:59.661149 23234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:59.661171 23234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:59.662366 23234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:59.662405 23234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:59.662431 23234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:06:59.663902 23234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab
--principal=oryx/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:59.664124 23234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:59.664337 23234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:59.669107 23246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:59.669083 23240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:59.669137 23249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:59.669365 23234 server_base.cc:1034] running on GCE node
I20260806 13:06:59.669603 23234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:59.669837 23234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:59.670972 23234 hybrid_clock.cc:648] HybridClock initialized: now 1786021619670951 us; error 35 us; skew 500 ppm
Aug 06 13:06:59 dist-test-slave-q9h9 krb5kdc[23065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021619, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:59.676084 23234 init.cc:377] Logged in from keytab as oryx/127.13.224.65@KRBTEST.COM (short username oryx)
I20260806 13:06:59.676499 23234 webserver.cc:460] Webserver started at http://127.13.224.65:35893/ using document root <none> and password file <none>
I20260806 13:06:59.676671 23234 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:59.676710 23234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:59.676803 23234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:59.677737 23234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "f6cf2c37a8844c168a3fdfc36a544799"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "87cc9c5acc1c85759507d3d619517e11"
server_key_iv: "9b0f377163ffea95211ba09cd21c4c8e"
server_key_version: "encryptionkey@0"
I20260806 13:06:59.678049 23234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "f6cf2c37a8844c168a3fdfc36a544799"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "87cc9c5acc1c85759507d3d619517e11"
server_key_iv: "9b0f377163ffea95211ba09cd21c4c8e"
server_key_version: "encryptionkey@0"
I20260806 13:06:59.679235 23234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:59.679868 23259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:59.680002 23234 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:06:59.680083 23234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "f6cf2c37a8844c168a3fdfc36a544799"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "87cc9c5acc1c85759507d3d619517e11"
server_key_iv: "9b0f377163ffea95211ba09cd21c4c8e"
server_key_version: "encryptionkey@0"
I20260806 13:06:59.680143 23234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:59.688349 23234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:59.689875 23234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:59.689999 23234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:59.690212 23234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:59.690497 23234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:59.690529 23234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:59.690555 23234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:59.690579 23234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:59.696039 23234 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:34533
I20260806 13:06:59.696092 23433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:34533 every 8 connection(s)
I20260806 13:06:59.696359 23234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
Aug 06 13:06:59 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021619, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
I20260806 13:06:59.702876 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 23234
I20260806 13:06:59.702970 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:06:59.703135 14209 external_mini_cluster.cc:1428] Setting key ade6b670e636af5fbf2df9fc337b543b
I20260806 13:06:59.704602 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:59.696955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:43615 (local address 127.13.224.126:35805)
0806 13:06:59.697025 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:59.697027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:59.697192 (+ 165us) server_negotiation.cc:407] Connection header received
0806 13:06:59.697622 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:59.697623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:59.697633 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:59.697679 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:59.698552 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:59.699045 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:59.699914 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:59.700095 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:59.700135 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:59.701948 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:59.701950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:59.701951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:59.701958 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:59.703300 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:59.703689 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:59.703689 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:59.703690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:59.703764 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:59.704006 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:59.704007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:59.704007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:59.704116 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:59.704171 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:59.704426 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 13:06:59.704511 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 13:06:59.704744 23437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:59.696931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.65:43615)
0806 13:06:59.697087 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:59.697120 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:59.697552 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:59.697729 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:59.697733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:59.698297 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:59.698476 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:59.698480 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:59.699102 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:59.699103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:59.699856 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:59.699859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:59.700144 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:59.700145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:59.700244 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:59.700621 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:59.700629 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:59.701866 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:59.703375 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:59.703384 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:59.703386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:59.703614 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:59.703823 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:59.703824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:59.703825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:59.703945 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:59.704171 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:59.704178 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:59.704310 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 13:06:59.704396 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 13:06:59.704542 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260806 13:06:59.705133 23434 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35805
I20260806 13:06:59.705224 23434 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:59.705408 23434 heartbeater.cc:511] Master 127.13.224.126:35805 requested a full tablet report, sending...
I20260806 13:06:59.705914 23137 ts_manager.cc:194] Registered new tserver with Master: f6cf2c37a8844c168a3fdfc36a544799 (127.13.224.65:34533)
I20260806 13:06:59.706512 23137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:43615
WARNING: no policy specified for oryx/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab.
Entry for principal oryx/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab.
I20260806 13:06:59.725677 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35805
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab
--principal=oryx/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:59.804872 23441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:59.805052 23441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:59.805074 23441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:59.806144 23441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:59.806174 23441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:59.806200 23441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:06:59.807559 23441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab
--principal=oryx/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:59.807725 23441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:59.807898 23441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:59.812459 23448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:59.812485 23451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:59.812513 23449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:59.812526 23441 server_base.cc:1034] running on GCE node
I20260806 13:06:59.812865 23441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:59.813046 23441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:59.814164 23441 hybrid_clock.cc:648] HybridClock initialized: now 1786021619814156 us; error 27 us; skew 500 ppm
Aug 06 13:06:59 dist-test-slave-q9h9 krb5kdc[23065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021619, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:59.816469 23441 init.cc:377] Logged in from keytab as oryx/127.13.224.66@KRBTEST.COM (short username oryx)
I20260806 13:06:59.816860 23441 webserver.cc:460] Webserver started at http://127.13.224.66:39469/ using document root <none> and password file <none>
I20260806 13:06:59.817046 23441 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:59.817091 23441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:59.817212 23441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:59.818202 23441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "29135061639e45c083246b11b65298a1"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "d3a051fec64f57f32d61fa93048edc33"
server_key_iv: "922a8bca3677a1f2a866795acf8d1779"
server_key_version: "encryptionkey@0"
I20260806 13:06:59.818518 23441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "29135061639e45c083246b11b65298a1"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "d3a051fec64f57f32d61fa93048edc33"
server_key_iv: "922a8bca3677a1f2a866795acf8d1779"
server_key_version: "encryptionkey@0"
I20260806 13:06:59.819681 23441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:06:59.820308 23465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:59.820453 23441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:06:59.820534 23441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "29135061639e45c083246b11b65298a1"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "d3a051fec64f57f32d61fa93048edc33"
server_key_iv: "922a8bca3677a1f2a866795acf8d1779"
server_key_version: "encryptionkey@0"
I20260806 13:06:59.820592 23441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:59.835556 23441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:59.837458 23441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:59.837585 23441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:59.837785 23441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:59.838084 23441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:59.838124 23441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:59.838150 23441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:59.838172 23441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:59.843539 23441 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:39637
I20260806 13:06:59.843598 23634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:39637 every 8 connection(s)
I20260806 13:06:59.843865 23441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
Aug 06 13:06:59 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021619, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
I20260806 13:06:59.849831 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 23441
I20260806 13:06:59.849923 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:06:59.850129 14209 external_mini_cluster.cc:1428] Setting key f98a7bd4ec657dd9074bd0b92ea4f619
I20260806 13:06:59.852320 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:59.844469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:46059 (local address 127.13.224.126:35805)
0806 13:06:59.844519 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 13:06:59.844522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:06:59.844718 (+ 196us) server_negotiation.cc:407] Connection header received
0806 13:06:59.845151 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:06:59.845153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:06:59.845163 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:06:59.845211 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:06:59.846042 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:59.846540 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:59.847440 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:06:59.847632 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:06:59.847685 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:59.849538 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:06:59.849539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:06:59.849540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:06:59.849547 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:06:59.851117 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:59.851498 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:59.851499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:59.851500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:59.851567 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:06:59.851763 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:06:59.851763 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:06:59.851764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:06:59.851880 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:06:59.851942 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 13:06:59.852159 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 13:06:59.852234 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:06:59.852389 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:06:59.844458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.66:46059)
0806 13:06:59.844607 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 13:06:59.844640 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 13:06:59.845065 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:06:59.845261 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:06:59.845264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:06:59.845779 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:06:59.845954 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:59.845959 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:59.846607 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:59.846607 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:59.847361 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:06:59.847364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:06:59.847690 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:06:59.847691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:06:59.847787 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:06:59.848188 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:06:59.848197 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:06:59.849460 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:06:59.851202 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:59.851208 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:59.851210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:59.851428 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:59.851626 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:06:59.851627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:06:59.851627 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:06:59.851712 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:06:59.851944 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:06:59.851948 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:06:59.852049 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 13:06:59.852124 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 13:06:59.852234 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260806 13:06:59.853046 23636 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35805
I20260806 13:06:59.853135 23636 heartbeater.cc:464] Registering TS with master...
I20260806 13:06:59.853298 23636 heartbeater.cc:511] Master 127.13.224.126:35805 requested a full tablet report, sending...
I20260806 13:06:59.853652 23137 ts_manager.cc:194] Registered new tserver with Master: 29135061639e45c083246b11b65298a1 (127.13.224.66:39637)
I20260806 13:06:59.854053 23137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:46059
WARNING: no policy specified for oryx/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab.
Entry for principal oryx/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab.
I20260806 13:06:59.872447 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35805
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab
--principal=oryx/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:06:59.952374 23642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:06:59.952556 23642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:06:59.952579 23642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:06:59.953668 23642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:06:59.953708 23642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:06:59.953735 23642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:06:59.955199 23642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab
--principal=oryx/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:06:59.955402 23642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:06:59.955603 23642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:06:59.960220 23648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:59.960265 23642 server_base.cc:1034] running on GCE node
W20260806 13:06:59.960220 23652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:06:59.960206 23647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:06:59.960577 23642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:06:59.960798 23642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:06:59.961927 23642 hybrid_clock.cc:648] HybridClock initialized: now 1786021619961911 us; error 30 us; skew 500 ppm
Aug 06 13:06:59 dist-test-slave-q9h9 krb5kdc[23065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021619, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:06:59.964226 23642 init.cc:377] Logged in from keytab as oryx/127.13.224.67@KRBTEST.COM (short username oryx)
I20260806 13:06:59.964582 23642 webserver.cc:460] Webserver started at http://127.13.224.67:44187/ using document root <none> and password file <none>
I20260806 13:06:59.964752 23642 fs_manager.cc:359] Metadata directory not provided
I20260806 13:06:59.964790 23642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:06:59.964876 23642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:06:59.965777 23642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "2ec05b2790c44deabe38bccef260ad4b"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "eeff27e79297d920a4a216a134cacefe"
server_key_iv: "309ed769a7656ba10f5246f51c931018"
server_key_version: "encryptionkey@0"
I20260806 13:06:59.966102 23642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "2ec05b2790c44deabe38bccef260ad4b"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "eeff27e79297d920a4a216a134cacefe"
server_key_iv: "309ed769a7656ba10f5246f51c931018"
server_key_version: "encryptionkey@0"
I20260806 13:06:59.967272 23642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:59.967947 23666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:59.968134 23642 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:06:59.968219 23642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "2ec05b2790c44deabe38bccef260ad4b"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "eeff27e79297d920a4a216a134cacefe"
server_key_iv: "309ed769a7656ba10f5246f51c931018"
server_key_version: "encryptionkey@0"
I20260806 13:06:59.968279 23642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:06:59.995817 23642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:06:59.997808 23642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:06:59.997916 23642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:06:59.998124 23642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:06:59.998399 23642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:06:59.998428 23642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:06:59.998448 23642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:06:59.998462 23642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.003656 23642 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:40499
I20260806 13:07:00.003729 23846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:40499 every 8 connection(s)
I20260806 13:07:00.003983 23642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:07:00.006649 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 23642
I20260806 13:07:00.006758 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:07:00.006930 14209 external_mini_cluster.cc:1428] Setting key c4d50dcdb8bdf30a8e883c8b1ee0e4d4
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021619, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
I20260806 13:07:00.012444 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.004629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:33389 (local address 127.13.224.126:35805)
0806 13:07:00.004683 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.004685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.004891 (+ 206us) server_negotiation.cc:407] Connection header received
0806 13:07:00.005342 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.005343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.005353 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.005404 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.006260 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.006910 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.008113 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.008312 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.008363 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.010190 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.010191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.010191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.010198 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.011443 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.011731 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.011731 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.011732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.011779 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.011962 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.011962 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.011963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.012076 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.012128 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.012320 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.012379 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 13:07:00.012593 23851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.004600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.67:33389)
0806 13:07:00.004771 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.004812 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.005268 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.005457 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.005460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.005979 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.006180 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.006185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.006999 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.007000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.008047 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.008050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.008369 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.008370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.008469 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.008836 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.008843 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.010103 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.011501 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.011504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.011505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.011682 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.011820 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.011820 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.011821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.011901 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.012122 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.012125 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.012224 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.012299 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.012407 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260806 13:07:00.013025 23848 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35805
I20260806 13:07:00.013108 23848 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:00.013280 23848 heartbeater.cc:511] Master 127.13.224.126:35805 requested a full tablet report, sending...
I20260806 13:07:00.013618 23137 ts_manager.cc:194] Registered new tserver with Master: 2ec05b2790c44deabe38bccef260ad4b (127.13.224.67:40499)
I20260806 13:07:00.013952 23137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:33389
I20260806 13:07:00.018458 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.224.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.224.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.224.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.125.keytab.
Entry for principal oryx/127.13.224.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.125.keytab.
I20260806 13:07:00.040520 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.125:34187
--webserver_interface=127.13.224.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.125.keytab
--principal=oryx/127.13.224.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:35805,127.13.224.125:34187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:00.119972 23856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:00.120165 23856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:00.120203 23856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:00.121438 23856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:07:00.121485 23856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:00.121506 23856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:00.121522 23856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:07:00.121542 23856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:07:00.123111 23856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:35805,127.13.224.125:34187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.125.keytab
--principal=oryx/127.13.224.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.125:34187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.224.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
W20260806 13:07:00.123235 23856 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.224.126:35805,127.13.224.125:34187'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 13:07:00.123385 23856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:00.123587 23856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:00.128171 23862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:00.128172 23866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:00.128227 23863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:00.128451 23856 server_base.cc:1034] running on GCE node
I20260806 13:07:00.128681 23856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:00.128947 23856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:00.130095 23856 hybrid_clock.cc:648] HybridClock initialized: now 1786021620130070 us; error 30 us; skew 500 ppm
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:00.132470 23856 init.cc:377] Logged in from keytab as oryx/127.13.224.125@KRBTEST.COM (short username oryx)
I20260806 13:07:00.132849 23856 webserver.cc:460] Webserver started at http://127.13.224.125:41637/ using document root <none> and password file <none>
I20260806 13:07:00.133047 23856 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:00.133085 23856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:00.133178 23856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:00.134172 23856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/data/instance:
uuid: "225e05dbd8e04293906d6f243481dfd6"
format_stamp: "Formatted at 2026-08-06 13:07:00 on dist-test-slave-q9h9"
server_key: "f0a65938e18744e945fc113bc3039644"
server_key_iv: "bc88b60c50ee9479603f8197dd4fc8cb"
server_key_version: "encryptionkey@0"
I20260806 13:07:00.134471 23856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/wal/instance:
uuid: "225e05dbd8e04293906d6f243481dfd6"
format_stamp: "Formatted at 2026-08-06 13:07:00 on dist-test-slave-q9h9"
server_key: "f0a65938e18744e945fc113bc3039644"
server_key_iv: "bc88b60c50ee9479603f8197dd4fc8cb"
server_key_version: "encryptionkey@0"
I20260806 13:07:00.135646 23856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:07:00.136267 23879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.136410 23856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:00.136493 23856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/wal
uuid: "225e05dbd8e04293906d6f243481dfd6"
format_stamp: "Formatted at 2026-08-06 13:07:00 on dist-test-slave-q9h9"
server_key: "f0a65938e18744e945fc113bc3039644"
server_key_iv: "bc88b60c50ee9479603f8197dd4fc8cb"
server_key_version: "encryptionkey@0"
I20260806 13:07:00.136551 23856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:00.163830 23856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:00.165964 23856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:00.166126 23856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:00.169852 23856 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.125:34187
I20260806 13:07:00.169888 23959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.125:34187 every 8 connection(s)
I20260806 13:07:00.170243 23856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/data/info.pb
I20260806 13:07:00.171267 23960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:00.172791 23960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:07:00.174666 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 23856
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.125@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.224.125@KRBTEST.COM
I20260806 13:07:00.182894 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.173444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46160 (local address 127.13.224.126:35805)
0806 13:07:00.173495 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.173497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.173730 (+ 233us) server_negotiation.cc:407] Connection header received
0806 13:07:00.174208 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.174209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.174223 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.174275 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.175055 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.175633 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.176888 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.177097 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.177149 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.179504 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.179505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.179506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.179513 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.181351 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.181852 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.181853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.181853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.181914 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.182203 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.182204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.182204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.182356 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.182421 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.182721 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.182801 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 13:07:00.183012 23966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.173416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46160)
0806 13:07:00.173585 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.173627 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.174110 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.174345 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.174349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.174673 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.174973 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.174980 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.175752 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.175753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.176802 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.176805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.177167 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.177168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.177295 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.177748 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.177756 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.179420 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.181430 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.181436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.181438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.181761 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.181986 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.181987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.181987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.182143 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.182430 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.182437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.182560 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.182640 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.182801 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260806 13:07:00.183481 23970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.175317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38350 (local address 127.13.224.125:34187)
0806 13:07:00.175469 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.175474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.175498 (+ 24us) server_negotiation.cc:407] Connection header received
0806 13:07:00.175539 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.175541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.175560 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.175645 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.176239 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.177137 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.177976 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.178428 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.178484 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.180773 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.180782 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.180782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.180789 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.182278 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.182581 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.182584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.182584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.182640 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.182808 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.182809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.182809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.182952 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.183106 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.183315 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.183394 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260806 13:07:00.183691 23960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.125@KRBTEST.COM for oryx/127.13.224.125@KRBTEST.COM
I20260806 13:07:00.189101 23966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.183940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34187 (local address 127.0.0.1:38366)
0806 13:07:00.183980 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.183990 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.184025 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.184178 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.184178 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.184317 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.184351 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.184352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.184894 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.184894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.185341 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.185342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.185691 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.185692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.185728 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.186151 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.186155 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.187188 (+ 1033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.188397 (+ 1209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.188398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.188398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.188546 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.188684 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.188685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.188685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.188725 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.188898 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.188899 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.188931 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.188974 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.189043 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 13:07:00.189193 23970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.183981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38366 (local address 127.13.224.125:34187)
0806 13:07:00.184012 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.184014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.184021 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:00.184082 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.184083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.184093 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.184171 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.184400 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.184830 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.185417 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.185632 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.185693 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.187275 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.187276 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.187276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.187282 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.188332 (+ 1050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.188601 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.188602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.188602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.188647 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.188761 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.188761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.188761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.188841 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.188881 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.188990 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.189097 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":15}
I20260806 13:07:00.190541 23960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: Bootstrap starting.
I20260806 13:07:00.191339 23960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:00.191596 23960 log.cc:824] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:00.192291 23960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: No bootstrap required, opened a new log
I20260806 13:07:00.193656 23960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } }
I20260806 13:07:00.193775 23960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:00.193807 23960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 225e05dbd8e04293906d6f243481dfd6, State: Initialized, Role: FOLLOWER
I20260806 13:07:00.193923 23960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } }
I20260806 13:07:00.194208 23960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:07:00.194175 23972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } } }
I20260806 13:07:00.194260 23972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 13:07:00.195719 23992 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:07:00.195773 23992 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:07:00.197305 23856 master_runner.cc:428] Detected that this master 225e05dbd8e04293906d6f243481dfd6 is joining an existing cluster
I20260806 13:07:00.197379 23856 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.224.125:34187
I20260806 13:07:00.204524 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.198416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46170 (local address 127.13.224.126:35805)
0806 13:07:00.198460 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.198462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.198642 (+ 180us) server_negotiation.cc:407] Connection header received
0806 13:07:00.198750 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.198751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.198762 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.198792 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.199148 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.199846 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.200883 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.201128 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.201182 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.202064 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.202065 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.202065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.202072 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.203384 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.203828 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.203829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.203829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.203887 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.204046 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.204047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.204047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.204165 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.204233 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.204396 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.204461 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:07:00.204555 24006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.198385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46170)
0806 13:07:00.198574 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.198585 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.198669 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.198857 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.198857 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.199002 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.199077 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.199078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.199926 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.199927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.200805 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.200808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.201187 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.201188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.201266 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.201622 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.201627 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.201970 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.203457 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.203458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.203458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.203769 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.203939 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.203939 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.203940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.203996 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.204234 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.204235 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.204279 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.204354 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.204450 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":424,"threads_started":1}
I20260806 13:07:00.205334 24010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.198431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34187 (local address 127.0.0.1:38372)
0806 13:07:00.198863 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.198874 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.198930 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.199067 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.199067 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.199253 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.199316 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.199318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.200085 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.200086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.200635 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.200637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.201026 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.201027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.201100 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.201513 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.201518 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.201967 (+ 449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.203873 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.203874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.203874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.204062 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.204249 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.204249 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.204250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.204313 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.204904 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.204905 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.204936 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.205133 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.205227 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
I20260806 13:07:00.205575 23970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.198414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38372 (local address 127.13.224.125:34187)
0806 13:07:00.198454 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.198456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.198977 (+ 521us) server_negotiation.cc:407] Connection header received
0806 13:07:00.198992 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.198993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.199000 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.199035 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.199389 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.200022 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.200705 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.200948 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.201008 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.202071 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.202072 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.202098 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.202105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.203794 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.204142 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.204143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.204143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.204187 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.204706 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.204707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.204708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.204830 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.204885 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.205438 (+ 553us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.205503 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021619, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.126@KRBTEST.COM for oryx/127.13.224.125@KRBTEST.COM
I20260806 13:07:00.213476 23970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.207017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38380 (local address 127.13.224.125:34187)
0806 13:07:00.207053 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.207055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.207174 (+ 119us) server_negotiation.cc:407] Connection header received
0806 13:07:00.207259 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.207259 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.207267 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.207307 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.207753 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.208156 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.209188 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.209369 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.209418 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.211381 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.211382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.211383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.211389 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.212502 (+ 1113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.212895 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.212896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.212896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.212936 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.213088 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.213088 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.213089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.213168 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.213209 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.213372 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.213423 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:07:00.213577 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.206952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34187 (local address 127.0.0.1:38380)
0806 13:07:00.207069 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.207116 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.207188 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.207314 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.207317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.207636 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.207690 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.207692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.208218 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.208219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.209121 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.209124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.209435 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.209437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.209509 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.209892 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.209896 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.211287 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.212571 (+ 1284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.212579 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.212582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.212833 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.212973 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.212974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.212974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.213034 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.213218 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.213223 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.213274 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.213358 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.213469 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":66,"threads_started":1}
I20260806 13:07:00.214107 23137 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.224.125:34187: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "225e05dbd8e04293906d6f243481dfd6"
member_type: NON_VOTER
last_known_addr {
host: "127.13.224.125"
port: 34187
}
attrs {
promote: true
}
}
dest_uuid: "bf238ca059c646dcaa34f2ee20cd1917"
cas_config_opid_index: -1
I20260806 13:07:00.214421 23137 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true } }
I20260806 13:07:00.214897 24013 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 225e05dbd8e04293906d6f243481dfd6 (127.13.224.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true } } }
I20260806 13:07:00.215198 23917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:00.215971 23917 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 FOLLOWER]: Refusing update from remote peer bf238ca059c646dcaa34f2ee20cd1917: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 13:07:00.215993 24013 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.224.125:34187
I20260806 13:07:00.216329 23856 master.cc:537] Master@127.13.224.125:34187 shutting down...
I20260806 13:07:00.216337 24012 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true } } }
I20260806 13:07:00.216522 24012 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:00.216334 24013 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 13:07:00.217497 23112 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 13:07:00.217969 23112 consensus_peers.cc:487] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 -> Peer 225e05dbd8e04293906d6f243481dfd6 (127.13.224.125:34187): Couldn't send request to peer 225e05dbd8e04293906d6f243481dfd6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 13:07:00.218859 23856 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:00.218946 23856 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:00.218992 23856 tablet_replica.cc:330] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: stopping tablet replica
I20260806 13:07:00.231518 23856 master.cc:552] Master@127.13.224.125:34187 shutdown complete.
I20260806 13:07:00.231571 23856 master_runner.cc:305] Clearing existing system tablet
I20260806 13:07:00.231702 23856 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:00.231753 23856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:00.232455 23856 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:00.232949 24018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.233064 23856 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:00.233129 23856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.233153 23856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/wal
uuid: "225e05dbd8e04293906d6f243481dfd6"
format_stamp: "Formatted at 2026-08-06 13:07:00 on dist-test-slave-q9h9"
server_key: "f0a65938e18744e945fc113bc3039644"
server_key_iv: "bc88b60c50ee9479603f8197dd4fc8cb"
server_key_version: "encryptionkey@0"
I20260806 13:07:00.234063 23856 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:00.235610 23856 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 13:07:00.235684 23856 log.cc:1196] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 13:07:00.235873 23856 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: Deleting consensus metadata
I20260806 13:07:00.235992 23856 master_runner.cc:315] Copying system tablet from 127.13.224.126:35805
I20260806 13:07:00.236615 23856 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: tablet copy: Beginning tablet copy session from remote peer at address 127.13.224.126:35805
I20260806 13:07:00.242779 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.236871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46172)
0806 13:07:00.237173 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.237182 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.237246 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.237377 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.237377 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.237525 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.237582 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.237584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.238569 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.238570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.239110 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.239112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.239539 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.239540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.239584 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.240023 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.240027 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.240314 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.241881 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.241882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.241883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.242079 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.242260 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.242261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.242261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.242312 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.242539 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.242540 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.242575 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.242620 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.242693 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":66,"threads_started":1}
I20260806 13:07:00.242822 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.236941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.13.224.126:35805)
0806 13:07:00.236990 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.236992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.237242 (+ 250us) server_negotiation.cc:407] Connection header received
0806 13:07:00.237290 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.237291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.237303 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.237373 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.237645 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.238494 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.239177 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.239479 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.239542 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.240395 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.240396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.240397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.240405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.241799 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.242137 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.242139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.242139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.242201 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.242360 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.242361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.242361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.242487 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.242552 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.242650 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.242728 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 13:07:00.243098 23168 tablet_copy_service.cc:140] P bf238ca059c646dcaa34f2ee20cd1917: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 225e05dbd8e04293906d6f243481dfd6 ({username='oryx', principal='oryx/127.13.224.125@KRBTEST.COM'} at 127.0.0.1:46172)
I20260806 13:07:00.243202 23168 tablet_copy_service.cc:161] P bf238ca059c646dcaa34f2ee20cd1917: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 225e05dbd8e04293906d6f243481dfd6 at {username='oryx', principal='oryx/127.13.224.125@KRBTEST.COM'} at 127.0.0.1:46172: session id = 225e05dbd8e04293906d6f243481dfd6-00000000000000000000000000000000
I20260806 13:07:00.244145 23168 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 13:07:00.244755 23856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 13:07:00.246060 23856 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: tablet copy: Starting download of 0 data blocks...
I20260806 13:07:00.246243 23856 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: tablet copy: Starting download of 1 WAL segments...
I20260806 13:07:00.247115 23856 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 13:07:00.247862 23168 tablet_copy_service.cc:342] P bf238ca059c646dcaa34f2ee20cd1917: Request end of tablet copy session 225e05dbd8e04293906d6f243481dfd6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.224.125@KRBTEST.COM'} at 127.0.0.1:46172
I20260806 13:07:00.247923 23168 tablet_copy_service.cc:433] P bf238ca059c646dcaa34f2ee20cd1917: ending tablet copy session 225e05dbd8e04293906d6f243481dfd6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 225e05dbd8e04293906d6f243481dfd6
I20260806 13:07:00.250304 23856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:00.251361 24030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:00.251490 24028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:00.251561 24032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:00.251722 23856 server_base.cc:1034] running on GCE node
I20260806 13:07:00.251813 23856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:00.251931 23856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:00.253042 23856 hybrid_clock.cc:648] HybridClock initialized: now 1786021620253026 us; error 24 us; skew 500 ppm
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:00.254243 23856 init.cc:377] Logged in from keytab as oryx/127.13.224.125@KRBTEST.COM (short username oryx)
I20260806 13:07:00.254565 23856 webserver.cc:460] Webserver started at http://127.13.224.125:34853/ using document root <none> and password file <none>
I20260806 13:07:00.254681 23856 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:00.254712 23856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:00.255256 23856 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:00.255728 24044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.255856 23856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.255913 23856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/wal
uuid: "225e05dbd8e04293906d6f243481dfd6"
format_stamp: "Formatted at 2026-08-06 13:07:00 on dist-test-slave-q9h9"
server_key: "f0a65938e18744e945fc113bc3039644"
server_key_iv: "bc88b60c50ee9479603f8197dd4fc8cb"
server_key_version: "encryptionkey@0"
I20260806 13:07:00.255964 23856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:00.263221 23856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:00.264858 23856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:00.267915 23856 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.125:34187
I20260806 13:07:00.268283 23856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-1/data/info.pb
I20260806 13:07:00.268303 24124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.125:34187 every 8 connection(s)
I20260806 13:07:00.269109 24127 sys_catalog.cc:263] Verifying existing consensus state
I20260806 13:07:00.269476 24127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: Bootstrap starting.
I20260806 13:07:00.271718 24127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:07:00.271930 24127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: Bootstrap complete.
I20260806 13:07:00.272089 24127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true } }
I20260806 13:07:00.272152 24127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 225e05dbd8e04293906d6f243481dfd6, State: Initialized, Role: LEARNER
I20260806 13:07:00.272185 24127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true } }
I20260806 13:07:00.272347 24127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:07:00.272332 24130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true } } }
I20260806 13:07:00.272409 24130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: This master's current role is: LEARNER
I20260806 13:07:00.273612 24151 catalog_manager.cc:1259] Loaded cluster ID: 757a651f8e0b4b919e896eb6b1b83831
I20260806 13:07:00.273653 24151 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: loading cluster ID for follower catalog manager: success
I20260806 13:07:00.274690 24151 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: acquiring CA information for follower catalog manager: success
I20260806 13:07:00.274961 24151 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 13:07:00.291002 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.285127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38396 (local address 127.13.224.125:34187)
0806 13:07:00.285230 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.285233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.285240 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:00.285261 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.285262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.285272 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.285332 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.285830 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.286542 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.287221 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.287427 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.287482 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.288614 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.288615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.288615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.288622 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.289966 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.290305 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.290306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.290306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.290359 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.290517 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.290518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.290519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.290616 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.290699 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.290825 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.290900 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":49,"threads_started":1}
WARNING: no policy specified for oryx/127.13.224.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.224.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.224.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.124.keytab.
Entry for principal oryx/127.13.224.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.124.keytab.
I20260806 13:07:00.314218 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.124:42299
--webserver_interface=127.13.224.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.124.keytab
--principal=oryx/127.13.224.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:35805,127.13.224.125:34187,127.13.224.124:42299
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:00.424254 24157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:00.424448 24157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:00.424472 24157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:00.425671 24157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:07:00.425716 24157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:00.425733 24157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:00.425763 24157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:07:00.425791 24157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:07:00.427487 24157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:35805,127.13.224.125:34187,127.13.224.124:42299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.124.keytab
--principal=oryx/127.13.224.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.124:42299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.224.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:00.427747 24157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:00.427975 24157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:00.433043 24164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:00.433034 24165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:00.433127 24168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:00.433382 24157 server_base.cc:1034] running on GCE node
I20260806 13:07:00.433611 24157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:00.433851 24157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:00.434998 24157 hybrid_clock.cc:648] HybridClock initialized: now 1786021620434978 us; error 34 us; skew 500 ppm
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:00.437968 24157 init.cc:377] Logged in from keytab as oryx/127.13.224.124@KRBTEST.COM (short username oryx)
I20260806 13:07:00.438380 24157 webserver.cc:460] Webserver started at http://127.13.224.124:37585/ using document root <none> and password file <none>
I20260806 13:07:00.438580 24157 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:00.438618 24157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:00.438710 24157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:00.439728 24157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/data/instance:
uuid: "f2fdbe7f433645b982cd7572a06f9ade"
format_stamp: "Formatted at 2026-08-06 13:07:00 on dist-test-slave-q9h9"
server_key: "61ea323705b7ee53d79d4df598d916dd"
server_key_iv: "4c90590ce0849b244d5e7873c9d8a2f6"
server_key_version: "encryptionkey@0"
I20260806 13:07:00.440026 24157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/wal/instance:
uuid: "f2fdbe7f433645b982cd7572a06f9ade"
format_stamp: "Formatted at 2026-08-06 13:07:00 on dist-test-slave-q9h9"
server_key: "61ea323705b7ee53d79d4df598d916dd"
server_key_iv: "4c90590ce0849b244d5e7873c9d8a2f6"
server_key_version: "encryptionkey@0"
I20260806 13:07:00.441262 24157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:07:00.442004 24177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.442207 24157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 13:07:00.442297 24157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/wal
uuid: "f2fdbe7f433645b982cd7572a06f9ade"
format_stamp: "Formatted at 2026-08-06 13:07:00 on dist-test-slave-q9h9"
server_key: "61ea323705b7ee53d79d4df598d916dd"
server_key_iv: "4c90590ce0849b244d5e7873c9d8a2f6"
server_key_version: "encryptionkey@0"
I20260806 13:07:00.442374 24157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:00.459079 24157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:00.460772 24157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:00.460933 24157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:00.472730 24157 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.124:42299
I20260806 13:07:00.472796 24256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.124:42299 every 8 connection(s)
I20260806 13:07:00.473121 24157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/data/info.pb
I20260806 13:07:00.474480 24257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:00.476634 24257 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:07:00.482076 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 24157
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.124@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
I20260806 13:07:00.486436 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.477461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.13.224.126:35805)
0806 13:07:00.477510 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.477512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.477797 (+ 285us) server_negotiation.cc:407] Connection header received
0806 13:07:00.478526 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.478528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.478541 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.478593 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.479289 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.479781 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.480825 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.481017 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.481075 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.483829 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.483831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.483831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.483839 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.485125 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.485495 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.485496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.485497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.485552 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.485789 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.485790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.485790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.485907 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.485965 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.486267 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.486348 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 13:07:00.486595 24260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.477331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46176)
0806 13:07:00.477669 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.477712 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.478425 (+ 713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.478661 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.478664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.478933 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.479199 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.479204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.479860 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.479861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.480741 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.480744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.481143 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.481144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.481327 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.481819 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.481827 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.483745 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.485207 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.485214 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.485216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.485427 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.485607 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.485608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.485608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.485724 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.485964 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.485971 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.486139 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.486236 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.486382 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":208,"threads_started":1}
I20260806 13:07:00.487309 24257 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.224.124@KRBTEST.COM
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.124@KRBTEST.COM for oryx/127.13.224.125@KRBTEST.COM
I20260806 13:07:00.491454 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.482620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49612 (local address 127.13.224.124:42299)
0806 13:07:00.482866 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.482871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.482894 (+ 23us) server_negotiation.cc:407] Connection header received
0806 13:07:00.482914 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.482916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.482953 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.483021 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.483617 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.484430 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.485177 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.485512 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.485573 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.488636 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.488644 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.488644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.488650 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.490378 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.490679 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.490689 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.490689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.490739 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.490902 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.490902 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.490903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.491028 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.491172 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.491248 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.491317 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":98,"threads_started":1}
I20260806 13:07:00.493544 24260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.487556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34187 (local address 127.0.0.1:38408)
0806 13:07:00.487595 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.487604 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.487645 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.487795 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.487796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.487927 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.488021 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.488023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.488703 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.488704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.489356 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.489359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.489721 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.489723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.489829 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.490164 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.490168 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.491377 (+ 1209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.492834 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.492835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.492836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.492991 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.493134 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.493134 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.493134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.493179 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.493352 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.493352 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.493384 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.493423 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.493488 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 13:07:00.493570 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.487631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38408 (local address 127.13.224.125:34187)
0806 13:07:00.487675 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.487677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.487685 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:07:00.487709 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.487710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.487725 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.487774 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.488092 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.488558 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.489434 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.489646 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.489706 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.491510 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.491511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.491512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.491520 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.492751 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.493055 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.493056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.493056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.493101 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.493220 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.493221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.493221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.493310 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.493355 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.493446 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.493510 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:07:00.493954 24257 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.124@KRBTEST.COM for oryx/127.13.224.124@KRBTEST.COM
I20260806 13:07:00.499496 24260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.494227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42299 (local address 127.0.0.1:49614)
0806 13:07:00.494261 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.494271 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.494308 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.494420 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.494420 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.494535 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.494559 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.494560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.495196 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.495197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.495654 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.495655 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.496010 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.496011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.496056 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.496342 (+ 286us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.496346 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.497415 (+ 1069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.498641 (+ 1226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.498642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.498642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.498788 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.498942 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.498942 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.498943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.498993 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.499185 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.499185 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.499250 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.499305 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.499396 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 13:07:00.499544 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.494268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49614 (local address 127.13.224.124:42299)
0806 13:07:00.494296 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.494298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.494305 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:00.494358 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.494358 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.494367 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.494416 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.494617 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.495128 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.495718 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.495952 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.496006 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.497499 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.497500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.497501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.497507 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.498568 (+ 1061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.498846 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.498847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.498848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.498893 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.499041 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.499042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.499042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.499135 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.499187 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.499334 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.499444 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":15}
I20260806 13:07:00.501017 24257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: Bootstrap starting.
I20260806 13:07:00.502351 24257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:00.502907 24257 log.cc:824] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:00.503895 24257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: No bootstrap required, opened a new log
I20260806 13:07:00.507054 24257 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } }
I20260806 13:07:00.507175 24257 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:00.507217 24257 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2fdbe7f433645b982cd7572a06f9ade, State: Initialized, Role: FOLLOWER
I20260806 13:07:00.507310 24257 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } }
I20260806 13:07:00.507745 24257 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:07:00.507788 24263 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } } }
I20260806 13:07:00.507910 24263 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: This master's current role is: FOLLOWER
W20260806 13:07:00.511075 24275 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:07:00.511124 24275 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:07:00.513147 24157 master_runner.cc:428] Detected that this master f2fdbe7f433645b982cd7572a06f9ade is joining an existing cluster
I20260806 13:07:00.513188 24157 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.224.124:42299
I20260806 13:07:00.520809 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.514189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49622 (local address 127.13.224.124:42299)
0806 13:07:00.514224 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.514226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.514428 (+ 202us) server_negotiation.cc:407] Connection header received
0806 13:07:00.514507 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.514508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.514515 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.514539 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.515130 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.515596 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.516565 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.516855 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.516930 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.518095 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.518097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.518097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.518104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.519651 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.520015 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.520016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.520016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.520065 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.520238 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.520239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.520239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.520350 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.520393 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.520654 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.520725 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:07:00.521090 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.514165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42299 (local address 127.0.0.1:49622)
0806 13:07:00.514368 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.514378 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.514448 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.514598 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.514598 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.514838 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.515042 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.515045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.515662 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.515663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.516474 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.516477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.516972 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.516973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.517059 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.517512 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.517516 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.517965 (+ 449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.519740 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.519741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.519741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.519937 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.520125 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.520125 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.520126 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.520174 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.520435 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.520436 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.520536 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.520878 (+ 342us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.520963 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
I20260806 13:07:00.524137 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.514192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.13.224.126:35805)
0806 13:07:00.514286 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.514288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.518218 (+ 3930us) server_negotiation.cc:407] Connection header received
0806 13:07:00.518269 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.518270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.518280 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.518325 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.518965 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.519441 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.520227 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.520414 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.520444 (+ 30us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.521822 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.521824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.521824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.521831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.523081 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.523421 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.523422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.523423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.523481 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.523641 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.523641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.523642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.523777 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.523832 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.523980 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.524047 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 13:07:00.524371 24285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.514505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46188)
0806 13:07:00.518153 (+ 3648us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.518163 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.518212 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.518369 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.518370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.518659 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.518867 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.518869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.519515 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.519515 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.520101 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.520103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.520503 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.520503 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.520585 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.520997 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.521002 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.521753 (+ 751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.523153 (+ 1400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.523154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.523155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.523359 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.523533 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.523533 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.523534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.523592 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.523833 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.523834 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.523889 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.523946 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.524041 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3623,"thread_start_us":103,"threads_started":1}
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021619, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.126@KRBTEST.COM for oryx/127.13.224.124@KRBTEST.COM
I20260806 13:07:00.536967 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.526111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49636 (local address 127.13.224.124:42299)
0806 13:07:00.526149 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.526151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.526159 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:07:00.526218 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.526219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.526227 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.526270 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.526502 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.526923 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.527548 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.527741 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.527792 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.529481 (+ 1689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.529482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.529483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.529489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.535952 (+ 6463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.536306 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.536307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.536308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.536363 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.536525 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.536526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.536526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.536625 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.536678 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.536812 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.536880 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:07:00.536974 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.526081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42299 (local address 127.0.0.1:49636)
0806 13:07:00.526116 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.526125 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.526164 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.526272 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.526272 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.526409 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.526440 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.526441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.526988 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.526989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.527481 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.527482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.527798 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.527799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.527863 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.528209 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.528212 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.529402 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.536030 (+ 6628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.536031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.536031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.536248 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.536418 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.536419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.536419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.536475 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.536677 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.536678 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.536723 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.536781 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.536883 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 13:07:00.537451 23137 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.224.124:42299: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade"
member_type: NON_VOTER
last_known_addr {
host: "127.13.224.124"
port: 42299
}
attrs {
promote: true
}
}
dest_uuid: "bf238ca059c646dcaa34f2ee20cd1917"
cas_config_opid_index: 5
I20260806 13:07:00.537706 23137 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } }
I20260806 13:07:00.538082 24013 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f2fdbe7f433645b982cd7572a06f9ade (127.13.224.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } } }
I20260806 13:07:00.538370 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.514240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38414 (local address 127.13.224.125:34187)
0806 13:07:00.514321 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.514323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.514744 (+ 421us) server_negotiation.cc:407] Connection header received
0806 13:07:00.514792 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.514793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.514814 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.514860 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.515176 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.515911 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.516813 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.517071 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.517145 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.535124 (+ 17979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.535125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.535125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.535133 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.536864 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.537452 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.537454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.537454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.537515 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.537786 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.537787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.537787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.537922 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.537987 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.538207 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.538290 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 13:07:00.538714 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.514199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34187 (local address 127.0.0.1:38414)
0806 13:07:00.514675 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.514687 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.514747 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.514862 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.514863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.515016 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.515076 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.515078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.515987 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.515988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.516745 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.516746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.517150 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.517151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.517244 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.534422 (+ 17178us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.534427 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.535014 (+ 587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.536974 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.536975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.536975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.537380 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.537646 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.537647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.537648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.537715 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.538045 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.538047 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.538089 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.538484 (+ 395us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.538580 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":62,"threads_started":1}
I20260806 13:07:00.539040 24013 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.224.124:42299
I20260806 13:07:00.539111 24013 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } } }
I20260806 13:07:00.539175 24013 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:00.539608 24157 master.cc:537] Master@127.13.224.124:42299 shutting down...
W20260806 13:07:00.540159 23109 consensus_peers.cc:487] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 -> Peer f2fdbe7f433645b982cd7572a06f9ade (127.13.224.124:42299): Couldn't send request to peer f2fdbe7f433645b982cd7572a06f9ade. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 13:07:00.543905 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.539475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34187 (local address 127.0.0.1:38430)
0806 13:07:00.540906 (+ 1431us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.540917 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.540962 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.541099 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.541100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.541101 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:00.541134 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.541136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.541745 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.541746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.542836 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.542840 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.543650 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.543651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.543714 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.543748 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.543804 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.543839 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1416}
I20260806 13:07:00.544229 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.539556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38430 (local address 127.13.224.125:34187)
0806 13:07:00.539599 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.539601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.540968 (+ 1367us) server_negotiation.cc:407] Connection header received
0806 13:07:00.541010 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.541011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.541034 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.541086 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:00.541179 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.541648 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.542919 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.543465 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.543533 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.543612 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.544130 (+ 518us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.544168 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:07:00.544312 24157 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:00.544391 24157 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:00.544440 24157 tablet_replica.cc:330] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: stopping tablet replica
I20260806 13:07:00.544850 24086 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 LEARNER]: Refusing update from remote peer bf238ca059c646dcaa34f2ee20cd1917: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 13:07:00.545061 24014 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 13:07:00.545712 24086 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f2fdbe7f433645b982cd7572a06f9ade (127.13.224.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } } }
I20260806 13:07:00.546944 24130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf238ca059c646dcaa34f2ee20cd1917. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } } }
I20260806 13:07:00.547041 24130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: This master's current role is: LEARNER
I20260806 13:07:00.546975 24290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: NON_VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: true } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } } }
I20260806 13:07:00.547202 24290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: This master's current role is: LEARNER
I20260806 13:07:00.550499 24014 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917: attempting to promote NON_VOTER 225e05dbd8e04293906d6f243481dfd6 to VOTER
I20260806 13:07:00.550663 24014 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } }
I20260806 13:07:00.551047 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.550956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42299 (local address 127.0.0.1:49648)
0806 13:07:00.550985 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.551021 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260806 13:07:00.551036 24086 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 LEARNER]: Refusing update from remote peer bf238ca059c646dcaa34f2ee20cd1917: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 13:07:00.551229 24013 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 13:07:00.551416 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.551229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42299 (local address 127.0.0.1:49654)
0806 13:07:00.551367 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.551393 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":101,"threads_started":1}
W20260806 13:07:00.551468 23109 consensus_peers.cc:487] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 -> Peer f2fdbe7f433645b982cd7572a06f9ade (127.13.224.124:42299): Couldn't send request to peer f2fdbe7f433645b982cd7572a06f9ade. Status: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42299: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 13:07:00.551887 24012 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 225e05dbd8e04293906d6f243481dfd6 (127.13.224.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } } }
I20260806 13:07:00.552599 24012 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } } }
I20260806 13:07:00.552627 24014 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } } }
I20260806 13:07:00.552688 24012 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:00.552709 24014 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:00.552925 24086 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 225e05dbd8e04293906d6f243481dfd6 (127.13.224.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } } }
I20260806 13:07:00.553773 24290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } } }
I20260806 13:07:00.553853 24290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:07:00.559620 24157 master.cc:552] Master@127.13.224.124:42299 shutdown complete.
I20260806 13:07:00.559660 24157 master_runner.cc:305] Clearing existing system tablet
I20260806 13:07:00.559783 24157 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:00.559824 24157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:00.560367 24157 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260806 13:07:00.560781 24295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.560881 24157 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:00.560932 24157 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.560954 24157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/wal
uuid: "f2fdbe7f433645b982cd7572a06f9ade"
format_stamp: "Formatted at 2026-08-06 13:07:00 on dist-test-slave-q9h9"
server_key: "61ea323705b7ee53d79d4df598d916dd"
server_key_iv: "4c90590ce0849b244d5e7873c9d8a2f6"
server_key_version: "encryptionkey@0"
I20260806 13:07:00.561535 24157 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:00.562727 24157 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 13:07:00.562798 24157 log.cc:1196] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 13:07:00.562959 24157 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: Deleting consensus metadata
I20260806 13:07:00.563061 24157 master_runner.cc:315] Copying system tablet from 127.13.224.126:35805
I20260806 13:07:00.563601 24157 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: tablet copy: Beginning tablet copy session from remote peer at address 127.13.224.126:35805
I20260806 13:07:00.569700 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.563911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46202 (local address 127.13.224.126:35805)
0806 13:07:00.563959 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.563961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.564026 (+ 65us) server_negotiation.cc:407] Connection header received
0806 13:07:00.564068 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.564069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.564079 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.564146 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.564495 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.565252 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.566049 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.566370 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.566438 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.567342 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.567343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.567344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.567351 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.568727 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.569034 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.569035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.569035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.569095 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.569249 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.569250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.569250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.569372 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.569441 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.569534 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.569608 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 13:07:00.569705 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.563847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46202)
0806 13:07:00.563961 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.563971 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.564027 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.564149 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.564149 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.564302 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.564427 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.564429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.565337 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.565340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.565952 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.565955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.566435 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.566436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.566484 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.566944 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.566948 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.567263 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.568811 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.568812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.568812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.568972 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.569157 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.569158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.569158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.569203 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.569428 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.569429 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.569464 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.569507 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.569582 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":67,"threads_started":1}
I20260806 13:07:00.570161 23168 tablet_copy_service.cc:140] P bf238ca059c646dcaa34f2ee20cd1917: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f2fdbe7f433645b982cd7572a06f9ade ({username='oryx', principal='oryx/127.13.224.124@KRBTEST.COM'} at 127.0.0.1:46202)
I20260806 13:07:00.570216 23168 tablet_copy_service.cc:161] P bf238ca059c646dcaa34f2ee20cd1917: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f2fdbe7f433645b982cd7572a06f9ade at {username='oryx', principal='oryx/127.13.224.124@KRBTEST.COM'} at 127.0.0.1:46202: session id = f2fdbe7f433645b982cd7572a06f9ade-00000000000000000000000000000000
I20260806 13:07:00.570676 23168 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 13:07:00.571158 24157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 13:07:00.572579 24157 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: tablet copy: Starting download of 0 data blocks...
I20260806 13:07:00.572715 24157 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: tablet copy: Starting download of 1 WAL segments...
I20260806 13:07:00.573447 24157 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 13:07:00.574349 23168 tablet_copy_service.cc:342] P bf238ca059c646dcaa34f2ee20cd1917: Request end of tablet copy session f2fdbe7f433645b982cd7572a06f9ade-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.224.124@KRBTEST.COM'} at 127.0.0.1:46202
I20260806 13:07:00.574404 23168 tablet_copy_service.cc:433] P bf238ca059c646dcaa34f2ee20cd1917: ending tablet copy session f2fdbe7f433645b982cd7572a06f9ade-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f2fdbe7f433645b982cd7572a06f9ade
I20260806 13:07:00.577075 24157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:00.578172 24310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:00.578293 24157 server_base.cc:1034] running on GCE node
W20260806 13:07:00.578197 24308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:00.578204 24307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:00.578504 24157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:00.578629 24157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:00.579751 24157 hybrid_clock.cc:648] HybridClock initialized: now 1786021620579743 us; error 25 us; skew 500 ppm
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:00.580911 24157 init.cc:377] Logged in from keytab as oryx/127.13.224.124@KRBTEST.COM (short username oryx)
I20260806 13:07:00.581144 24157 webserver.cc:460] Webserver started at http://127.13.224.124:44079/ using document root <none> and password file <none>
I20260806 13:07:00.581226 24157 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:00.581252 24157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:00.581810 24157 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.582293 24319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.582397 24157 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.582440 24157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/wal
uuid: "f2fdbe7f433645b982cd7572a06f9ade"
format_stamp: "Formatted at 2026-08-06 13:07:00 on dist-test-slave-q9h9"
server_key: "61ea323705b7ee53d79d4df598d916dd"
server_key_iv: "4c90590ce0849b244d5e7873c9d8a2f6"
server_key_version: "encryptionkey@0"
I20260806 13:07:00.582491 24157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:00.592868 24157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:00.594517 24157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:00.599752 24157 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.124:42299
I20260806 13:07:00.599773 24396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.124:42299 every 8 connection(s)
I20260806 13:07:00.600122 24157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/master-2/data/info.pb
I20260806 13:07:00.600946 24397 sys_catalog.cc:263] Verifying existing consensus state
I20260806 13:07:00.601464 24397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: Bootstrap starting.
I20260806 13:07:00.604599 24397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:07:00.604830 24397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: Bootstrap complete.
I20260806 13:07:00.605026 24397 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } }
I20260806 13:07:00.605131 24397 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f2fdbe7f433645b982cd7572a06f9ade, State: Initialized, Role: LEARNER
I20260806 13:07:00.605183 24397 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } }
I20260806 13:07:00.605366 24397 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:07:00.605358 24400 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } } }
I20260806 13:07:00.605469 24400 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: This master's current role is: LEARNER
I20260806 13:07:00.607323 24416 catalog_manager.cc:1259] Loaded cluster ID: 757a651f8e0b4b919e896eb6b1b83831
I20260806 13:07:00.607365 24416 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: loading cluster ID for follower catalog manager: success
I20260806 13:07:00.608139 24416 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: acquiring CA information for follower catalog manager: success
I20260806 13:07:00.608393 24416 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 13:07:00.655473 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.648574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49672 (local address 127.13.224.124:42299)
0806 13:07:00.648720 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.648722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.648729 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:00.648760 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.648761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.648770 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.648836 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.649396 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.650211 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.650911 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.651172 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.651237 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.652916 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.652918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.652919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.652926 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.654403 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.654786 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.654787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.654787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.654848 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.655033 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.655034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.655034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.655142 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.655214 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.655340 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.655402 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260806 13:07:00.656080 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 23234
I20260806 13:07:00.674683 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:34533
--local_ip_for_outbound_sockets=127.13.224.65
--tserver_master_addrs=127.13.224.126:35805
--webserver_port=35893
--webserver_interface=127.13.224.65
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab
--principal=oryx/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.224.126:35805,127.13.224.125:34187
--tserver_master_addrs=127.13.224.126:35805,127.13.224.125:34187,127.13.224.124:42299 with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:00.755568 24418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:00.755761 24418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:00.755792 24418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:00.756917 24418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:00.756958 24418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:00.756991 24418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:07:00.758486 24418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab
--principal=oryx/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:34533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=35893
--tserver_master_addrs=127.13.224.126:35805,127.13.224.125:34187,127.13.224.124:42299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:00.758725 24418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:00.758935 24418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:00.763770 24428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:00.763767 24433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:00.763797 24426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:00.763976 24418 server_base.cc:1034] running on GCE node
I20260806 13:07:00.764168 24418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:00.764387 24418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:00.765524 24418 hybrid_clock.cc:648] HybridClock initialized: now 1786021620765517 us; error 28 us; skew 500 ppm
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:00.767807 24418 init.cc:377] Logged in from keytab as oryx/127.13.224.65@KRBTEST.COM (short username oryx)
I20260806 13:07:00.768173 24418 webserver.cc:460] Webserver started at http://127.13.224.65:35893/ using document root <none> and password file <none>
I20260806 13:07:00.768353 24418 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:00.768406 24418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:00.769721 24418 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:07:00.770283 24442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.770430 24418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:00.770511 24418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "f6cf2c37a8844c168a3fdfc36a544799"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "87cc9c5acc1c85759507d3d619517e11"
server_key_iv: "9b0f377163ffea95211ba09cd21c4c8e"
server_key_version: "encryptionkey@0"
I20260806 13:07:00.771030 24418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:00.778746 24418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:00.780721 24418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:00.780827 24418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:00.780995 24418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:00.781239 24418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:00.781268 24418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.781286 24418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:00.781302 24418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.787096 24418 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:34533
I20260806 13:07:00.787153 24608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:34533 every 8 connection(s)
I20260806 13:07:00.787467 24418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:07:00.789430 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 24418
I20260806 13:07:00.789592 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 23441
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
I20260806 13:07:00.804028 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.788425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:33335 (local address 127.13.224.126:35805)
0806 13:07:00.788471 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.788473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.788484 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:07:00.789036 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.789038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.789052 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.789110 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.795095 (+ 5985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.795834 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.797270 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.797654 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.797748 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.800457 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.800459 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.800460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.800469 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.802223 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.802858 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.802859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.802860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.802930 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.803262 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.803263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.803264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.803395 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.803457 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.803875 (+ 418us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.803942 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:07:00.804383 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.788100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.65:33335)
0806 13:07:00.788280 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.788322 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.788920 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.789183 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.789188 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.794605 (+ 5417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.794973 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.794982 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.795950 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.795951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.797179 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.797182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.797748 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.797749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.797920 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.798519 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.798529 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.800353 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.802332 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.802343 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.802346 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.802746 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.803017 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.803017 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.803018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.803186 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.803471 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.803480 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.803708 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.803866 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.804082 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
I20260806 13:07:00.804890 24613 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35805
I20260806 13:07:00.804994 24613 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:00.805161 24613 heartbeater.cc:511] Master 127.13.224.126:35805 requested a full tablet report, sending...
I20260806 13:07:00.805696 23137 ts_manager.cc:194] Re-registered known tserver with Master: f6cf2c37a8844c168a3fdfc36a544799 (127.13.224.65:34533)
I20260806 13:07:00.806377 23137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:33335
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.124@KRBTEST.COM
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.125@KRBTEST.COM
I20260806 13:07:00.815526 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:39637
--local_ip_for_outbound_sockets=127.13.224.66
--tserver_master_addrs=127.13.224.126:35805
--webserver_port=39469
--webserver_interface=127.13.224.66
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab
--principal=oryx/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.224.126:35805,127.13.224.125:34187
--tserver_master_addrs=127.13.224.126:35805,127.13.224.125:34187,127.13.224.124:42299 with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:07:00.816704 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.788140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:59933 (local address 127.13.224.124:42299)
0806 13:07:00.788207 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.788209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.789052 (+ 843us) server_negotiation.cc:407] Connection header received
0806 13:07:00.789135 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.789136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.789151 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.789203 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.809700 (+ 20497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.810363 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.811349 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.811630 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.811668 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.813838 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.813840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.813840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.813849 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.815541 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.815886 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.815887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.815888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.815947 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.816132 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.816133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.816133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.816266 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.816303 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.816508 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.816602 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 13:07:00.816875 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.788443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:57279 (local address 127.13.224.125:34187)
0806 13:07:00.788492 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.788493 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.788734 (+ 241us) server_negotiation.cc:407] Connection header received
0806 13:07:00.789168 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.789169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.789188 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.789233 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.809820 (+ 20587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.810582 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.811717 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.811907 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.811957 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.814283 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.814285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.814285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.814293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.815752 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.816080 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.816081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.816082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.816137 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.816322 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.816323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.816323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.816458 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.816522 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.816668 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.816742 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:07:00.816988 24621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.788101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42299 (local address 127.13.224.65:59933)
0806 13:07:00.788991 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.789000 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.789077 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.789210 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.789211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.809538 (+ 20327us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.809604 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.809607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.810454 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.810455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.811249 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.811252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.811742 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.811743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.811833 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.812364 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.812369 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.813736 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.815633 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.815634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.815635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.815834 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.816002 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.816003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.816003 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.816059 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.816352 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.816353 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.816395 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.816791 (+ 396us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.816891 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":70,"thread_start_us":102,"threads_started":1}
I20260806 13:07:00.817371 24609 heartbeater.cc:347] Connected to a master server at 127.13.224.124:42299
I20260806 13:07:00.817435 24609 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:00.817443 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.788101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34187 (local address 127.13.224.65:57279)
0806 13:07:00.788665 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.788675 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.788928 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.789277 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.789278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.809565 (+ 20287us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.809667 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.809681 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.810690 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.810691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.811619 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.811621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.811965 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.811965 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.812065 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.812472 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.812476 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.814210 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.815816 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.815817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.815817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.816009 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.816199 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.816199 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.816200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.816263 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.816518 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.816519 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.816570 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.816636 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.816727 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":2,"thread_start_us":110,"threads_started":1}
I20260806 13:07:00.817531 24609 heartbeater.cc:511] Master 127.13.224.124:42299 requested a full tablet report, sending...
I20260806 13:07:00.817797 24614 heartbeater.cc:347] Connected to a master server at 127.13.224.125:34187
I20260806 13:07:00.817844 24614 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:00.817883 24614 heartbeater.cc:511] Master 127.13.224.125:34187 requested a full tablet report, sending...
I20260806 13:07:00.818168 24345 ts_manager.cc:194] Registered new tserver with Master: f6cf2c37a8844c168a3fdfc36a544799 (127.13.224.65:34533)
I20260806 13:07:00.818261 24072 ts_manager.cc:194] Registered new tserver with Master: f6cf2c37a8844c168a3fdfc36a544799 (127.13.224.65:34533)
W20260806 13:07:00.897231 24623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:00.897426 24623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:00.897456 24623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:00.898644 24623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:00.898685 24623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:00.898720 24623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:07:00.900161 24623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab
--principal=oryx/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:39637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=39469
--tserver_master_addrs=127.13.224.126:35805,127.13.224.125:34187,127.13.224.124:42299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:00.900394 24623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:00.900601 24623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:00.905866 24639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:00.905884 24623 server_base.cc:1034] running on GCE node
W20260806 13:07:00.906041 24632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:00.905920 24636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:00.906287 24623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:00.906519 24623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:00.907650 24623 hybrid_clock.cc:648] HybridClock initialized: now 1786021620907640 us; error 27 us; skew 500 ppm
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:00.909934 24623 init.cc:377] Logged in from keytab as oryx/127.13.224.66@KRBTEST.COM (short username oryx)
I20260806 13:07:00.910392 24623 webserver.cc:460] Webserver started at http://127.13.224.66:39469/ using document root <none> and password file <none>
I20260806 13:07:00.910585 24623 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:00.910647 24623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:00.911911 24623 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:07:00.912464 24653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.912623 24623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:00.912704 24623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "29135061639e45c083246b11b65298a1"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "d3a051fec64f57f32d61fa93048edc33"
server_key_iv: "922a8bca3677a1f2a866795acf8d1779"
server_key_version: "encryptionkey@0"
I20260806 13:07:00.913012 24623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:00.921438 24623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:00.923090 24623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:00.923228 24623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:00.923421 24623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:00.923692 24623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:00.923724 24623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.923749 24623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:00.923775 24623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:00.929386 24623 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:39637
I20260806 13:07:00.929442 24829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:39637 every 8 connection(s)
I20260806 13:07:00.929729 24623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:07:00.931103 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 24623
I20260806 13:07:00.931262 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 23642
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.124@KRBTEST.COM
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
Aug 06 13:07:00 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021620, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.125@KRBTEST.COM
I20260806 13:07:00.941933 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:40499
--local_ip_for_outbound_sockets=127.13.224.67
--tserver_master_addrs=127.13.224.126:35805
--webserver_port=44187
--webserver_interface=127.13.224.67
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab
--principal=oryx/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.224.126:35805,127.13.224.125:34187
--tserver_master_addrs=127.13.224.126:35805,127.13.224.125:34187,127.13.224.124:42299 with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:07:00.942662 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.930470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:54875 (local address 127.13.224.124:42299)
0806 13:07:00.930516 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.930518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.930997 (+ 479us) server_negotiation.cc:407] Connection header received
0806 13:07:00.931366 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.931368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.931380 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.931409 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.932382 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.933355 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.935036 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.935355 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.935394 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.938919 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.938921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.938921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.938930 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.941087 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.941535 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.941536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.941537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.941596 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.941914 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.941916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.941916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.942089 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.942136 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.942491 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.942571 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:07:00.942744 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.930392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:51919 (local address 127.13.224.126:35805)
0806 13:07:00.930545 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.930547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.930851 (+ 304us) server_negotiation.cc:407] Connection header received
0806 13:07:00.931398 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.931399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.931410 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.931439 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.932589 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.933191 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.935464 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.935750 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.935797 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.938916 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.938917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.938917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.938925 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.941387 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.941931 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.941932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.941933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.941986 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.942217 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.942218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.942218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.942342 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.942382 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.942577 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.942654 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260806 13:07:00.942772 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.930489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:49041 (local address 127.13.224.125:34187)
0806 13:07:00.930581 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.930583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.930829 (+ 246us) server_negotiation.cc:407] Connection header received
0806 13:07:00.931357 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.931359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.931372 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.931405 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:00.933850 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.934593 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.937497 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.937771 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.937842 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.939930 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:00.939931 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:00.939931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:00.939939 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:00.941387 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.941819 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.941820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.941821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.941876 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:00.942265 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:00.942266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:00.942266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:00.942386 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:00.942424 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.942601 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.942668 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 13:07:00.943264 24842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.930370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.66:51919)
0806 13:07:00.930699 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.930747 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.931235 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.931581 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.931585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.932330 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.932509 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.932511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.933395 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.933396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.935370 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.935373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.935878 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.935879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.935972 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.936920 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.936925 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.938805 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.941477 (+ 2672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.941478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.941479 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.941852 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.942080 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.942080 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.942081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.942135 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.942428 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.942429 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.942468 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.942854 (+ 386us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.943094 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":964,"thread_start_us":84,"threads_started":1}
I20260806 13:07:00.943269 24838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.930372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42299 (local address 127.13.224.66:54875)
0806 13:07:00.930906 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.930929 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.931267 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.931978 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.931979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.932190 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.932270 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.932273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.933465 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.933466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.934938 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.934941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.935481 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.935482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.935665 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.936588 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.936598 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.938808 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.941178 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.941185 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.941187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.941475 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.941662 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.941663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.941663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.941827 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.942181 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.942189 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.942351 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.942790 (+ 439us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.943016 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":329,"thread_start_us":62,"threads_started":1}
I20260806 13:07:00.943884 24830 heartbeater.cc:347] Connected to a master server at 127.13.224.124:42299
I20260806 13:07:00.943920 24834 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35805
I20260806 13:07:00.943984 24830 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:00.944029 24834 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:00.944167 24830 heartbeater.cc:511] Master 127.13.224.124:42299 requested a full tablet report, sending...
I20260806 13:07:00.944168 24834 heartbeater.cc:511] Master 127.13.224.126:35805 requested a full tablet report, sending...
I20260806 13:07:00.944502 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.930372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34187 (local address 127.13.224.66:49041)
0806 13:07:00.930710 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.930741 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.931235 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.931581 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.931585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.931908 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:00.932147 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.932158 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.936188 (+ 4030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.936189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.936816 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.936818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.937930 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.937932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.938058 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.938497 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:00.938502 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:00.939852 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:00.941477 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.941478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.941479 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.941738 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.941945 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:00.941945 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:00.941945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:00.942192 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:00.942467 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:00.942468 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:00.942500 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.943261 (+ 761us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.943796 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260806 13:07:00.944566 23137 ts_manager.cc:194] Re-registered known tserver with Master: 29135061639e45c083246b11b65298a1 (127.13.224.66:39637)
I20260806 13:07:00.944634 24345 ts_manager.cc:194] Registered new tserver with Master: 29135061639e45c083246b11b65298a1 (127.13.224.66:39637)
I20260806 13:07:00.945004 24836 heartbeater.cc:347] Connected to a master server at 127.13.224.125:34187
I20260806 13:07:00.945022 23137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:51919
I20260806 13:07:00.945051 24836 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:00.945102 24836 heartbeater.cc:511] Master 127.13.224.125:34187 requested a full tablet report, sending...
I20260806 13:07:00.945393 24072 ts_manager.cc:194] Registered new tserver with Master: 29135061639e45c083246b11b65298a1 (127.13.224.66:39637)
I20260806 13:07:00.960043 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.957109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49678 (local address 127.13.224.124:42299)
0806 13:07:00.957154 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:00.957156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:00.957168 (+ 12us) server_negotiation.cc:407] Connection header received
0806 13:07:00.957223 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:00.957224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:00.957245 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:00.957303 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:00.957414 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.957882 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.959340 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.959751 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.959820 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.959871 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:00.959963 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 13:07:00.959990 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:07:00.960069 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:00.957049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42299 (local address 127.0.0.1:49678)
0806 13:07:00.957111 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:00.957123 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:00.957171 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:00.957313 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:00.957314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:00.957315 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:00.957360 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.957362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.957943 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.957943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.959240 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:00.959244 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:00.959828 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:00.959829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:00.959897 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:00.959901 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:07:00.959952 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 13:07:00.959995 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 13:07:00.960410 24360 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 13:07:00.960584 24400 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf238ca059c646dcaa34f2ee20cd1917. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } } }
I20260806 13:07:00.960697 24400 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: This master's current role is: LEARNER
W20260806 13:07:01.025878 24844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:01.026108 24844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:01.026134 24844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:01.027248 24844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:01.027282 24844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:01.027307 24844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:07:01.028788 24844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab
--principal=oryx/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:40499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=44187
--tserver_master_addrs=127.13.224.126:35805,127.13.224.125:34187,127.13.224.124:42299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:01.028970 24844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:01.029140 24844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:01.034077 24855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:01.034076 24850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:01.034183 24851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:01.034323 24844 server_base.cc:1034] running on GCE node
I20260806 13:07:01.034521 24844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:01.034752 24844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:01.035917 24844 hybrid_clock.cc:648] HybridClock initialized: now 1786021621035901 us; error 30 us; skew 500 ppm
Aug 06 13:07:01 dist-test-slave-q9h9 krb5kdc[23065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021621, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:01.038359 24844 init.cc:377] Logged in from keytab as oryx/127.13.224.67@KRBTEST.COM (short username oryx)
I20260806 13:07:01.038793 24844 webserver.cc:460] Webserver started at http://127.13.224.67:44187/ using document root <none> and password file <none>
I20260806 13:07:01.039012 24844 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:01.039093 24844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:01.040421 24844 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:01.040995 24872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:01.041188 24844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:01.041270 24844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "2ec05b2790c44deabe38bccef260ad4b"
format_stamp: "Formatted at 2026-08-06 13:06:59 on dist-test-slave-q9h9"
server_key: "eeff27e79297d920a4a216a134cacefe"
server_key_iv: "309ed769a7656ba10f5246f51c931018"
server_key_version: "encryptionkey@0"
I20260806 13:07:01.041580 24844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:01.054171 24844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:01.055936 24844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:01.056066 24844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:01.056281 24844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:01.056547 24844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:01.056581 24844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:01.056607 24844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:01.056624 24844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:01.062124 24844 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:40499
I20260806 13:07:01.062184 25057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:40499 every 8 connection(s)
I20260806 13:07:01.062482 24844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:07:01.067950 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 24844
Aug 06 13:07:01 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021621, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
Aug 06 13:07:01 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021621, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.125@KRBTEST.COM
Aug 06 13:07:01 dist-test-slave-q9h9 krb5kdc[23065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021621, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.124@KRBTEST.COM
I20260806 13:07:01.073004 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.063332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:46207 (local address 127.13.224.125:34187)
0806 13:07:01.063381 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.063383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.064262 (+ 879us) server_negotiation.cc:407] Connection header received
0806 13:07:01.064338 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.064339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.064351 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.064409 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:01.065001 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.065753 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.066781 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.067072 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.067146 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.069848 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:01.069850 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:01.069850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:01.069859 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:01.071382 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.071833 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.071835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.071835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.071896 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.072159 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.072160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.072160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.072291 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:01.072407 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.072838 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.072911 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:07:01.073098 25063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.063110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.67:46341)
0806 13:07:01.063428 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:01.063466 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:01.063941 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:01.064220 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:01.064224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:01.064742 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:01.064866 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.064871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.065830 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.065831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.066706 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.066707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.067211 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.067212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.067272 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.067703 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:01.067713 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:01.069303 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:01.071857 (+ 2554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:01.071858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:01.071858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:01.072037 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:01.072226 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:01.072227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:01.072227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:01.072276 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:01.072578 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:01.072579 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:01.072622 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 13:07:01.072684 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 13:07:01.072866 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":27,"spinlock_wait_cycles":7552,"thread_start_us":198,"threads_started":1}
I20260806 13:07:01.073302 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.063111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42299 (local address 127.13.224.67:42785)
0806 13:07:01.063534 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:01.063543 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:01.063973 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:01.064221 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:01.064225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:01.064653 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:01.064866 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.064872 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.065567 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.065568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.066704 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.066706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.067120 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.067121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.067265 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.068568 (+ 1303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:01.068573 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:01.070386 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:01.072109 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:01.072110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:01.072111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:01.072360 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:01.072589 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:01.072590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:01.072590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:01.072650 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:01.072987 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:01.072988 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:01.073029 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 13:07:01.073088 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 13:07:01.073183 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":11,"thread_start_us":105,"threads_started":1}
I20260806 13:07:01.073531 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.063272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:46341 (local address 127.13.224.126:35805)
0806 13:07:01.063319 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.063321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.063550 (+ 229us) server_negotiation.cc:407] Connection header received
0806 13:07:01.064050 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.064051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.064065 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.064131 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:01.064954 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.065749 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.066818 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.067107 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.067160 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.069406 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:01.069407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:01.069408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:01.069415 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:01.071785 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.072109 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.072110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.072111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.072164 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.072376 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.072377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.072377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.072499 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:01.072553 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.073348 (+ 795us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.073446 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:07:01.073694 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.063228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:42785 (local address 127.13.224.124:42299)
0806 13:07:01.063273 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.063275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.063631 (+ 356us) server_negotiation.cc:407] Connection header received
0806 13:07:01.064078 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.064080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.064094 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.064148 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:01.064977 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.065469 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.066812 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.067015 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.067049 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.070488 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:01.070490 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:01.070490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:01.070498 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:01.072029 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.072457 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.072458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.072459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.072521 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.072731 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.072732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.072732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.072870 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:01.072928 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.073526 (+ 598us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.073604 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:07:01.073909 25060 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35805
I20260806 13:07:01.074002 25060 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:01.074123 25058 heartbeater.cc:347] Connected to a master server at 127.13.224.124:42299
I20260806 13:07:01.074184 25058 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:01.074206 25060 heartbeater.cc:511] Master 127.13.224.126:35805 requested a full tablet report, sending...
I20260806 13:07:01.074265 25058 heartbeater.cc:511] Master 127.13.224.124:42299 requested a full tablet report, sending...
I20260806 13:07:01.074349 25065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.063111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34187 (local address 127.13.224.67:46207)
0806 13:07:01.064080 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:01.064089 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:01.064284 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:01.064412 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:01.064412 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:01.064623 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:01.064867 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.064872 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.066081 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.066082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.066700 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.066702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.067154 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.067155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.067265 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.067908 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:01.067913 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:01.069745 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:01.071469 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:01.071476 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:01.071478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:01.071751 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:01.071959 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:01.071960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:01.071960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:01.072077 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:01.072425 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:01.072432 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:01.072582 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 13:07:01.072684 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 13:07:01.074135 (+ 1451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":943,"mutex_wait_us":1289,"spinlock_wait_cycles":12544,"thread_start_us":271,"threads_started":1}
I20260806 13:07:01.074647 23137 ts_manager.cc:194] Re-registered known tserver with Master: 2ec05b2790c44deabe38bccef260ad4b (127.13.224.67:40499)
I20260806 13:07:01.074666 25061 heartbeater.cc:347] Connected to a master server at 127.13.224.125:34187
I20260806 13:07:01.074705 25061 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:01.074757 25061 heartbeater.cc:511] Master 127.13.224.125:34187 requested a full tablet report, sending...
I20260806 13:07:01.075062 24072 ts_manager.cc:194] Registered new tserver with Master: 2ec05b2790c44deabe38bccef260ad4b (127.13.224.67:40499)
I20260806 13:07:01.075130 23137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:46341
I20260806 13:07:01.075407 24345 ts_manager.cc:194] Registered new tserver with Master: 2ec05b2790c44deabe38bccef260ad4b (127.13.224.67:40499)
I20260806 13:07:01.081233 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.069275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46216 (local address 127.13.224.126:35805)
0806 13:07:01.070888 (+ 1613us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.070890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.070899 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:01.071259 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.071260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.071272 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.074934 (+ 3662us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:01.074965 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.075630 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.076465 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.076671 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.076720 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.078118 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:01.078119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:01.078120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:01.078127 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:01.079939 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.080348 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.080349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.080350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.080411 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.080607 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.080608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.080608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.080736 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:01.080833 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.081079 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.081168 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1557,"thread_start_us":51,"threads_started":1}
I20260806 13:07:01.081712 25075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.069867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49682 (local address 127.13.224.124:42299)
0806 13:07:01.071128 (+ 1261us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.071130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.071137 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:01.071154 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.071154 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.071165 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.071218 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:01.073488 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.075976 (+ 2488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.077196 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.077501 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.077557 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.078869 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:01.078871 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:01.078871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:01.078878 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:01.080383 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.080792 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.080794 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.080794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.080851 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.081016 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.081017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.081017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.081226 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:01.081320 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.081450 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.081621 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1243,"thread_start_us":69,"threads_started":1}
I20260806 13:07:01.081799 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.070306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38442 (local address 127.13.224.125:34187)
0806 13:07:01.070630 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.070632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.070971 (+ 339us) server_negotiation.cc:407] Connection header received
0806 13:07:01.070995 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.070996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.071009 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.071061 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:01.072699 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.073400 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.077196 (+ 3796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.077501 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.077560 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.079041 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:01.079043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:01.079043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:01.079049 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:01.080553 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.080863 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.080865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.080865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.080922 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.081092 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.081093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.081094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.081226 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:01.081320 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.081630 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.081713 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":64,"threads_started":1}
I20260806 13:07:01.083104 23137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:07:01.084745 23137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:07:01.085449 24013 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: NON_VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: true } } }
I20260806 13:07:01.085533 24013 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:01.087069 24013 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917: attempting to promote NON_VOTER f2fdbe7f433645b982cd7572a06f9ade to VOTER
I20260806 13:07:01.087199 24013 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } }
I20260806 13:07:01.087565 24086 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 FOLLOWER]: Refusing update from remote peer bf238ca059c646dcaa34f2ee20cd1917: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 13:07:01.087741 25084 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 13:07:01.087956 24360 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 1 LEARNER]: Refusing update from remote peer bf238ca059c646dcaa34f2ee20cd1917: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 13:07:01.088165 25084 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 13:07:01.088290 25084 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f2fdbe7f433645b982cd7572a06f9ade (127.13.224.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } } }
I20260806 13:07:01.088521 24086 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f2fdbe7f433645b982cd7572a06f9ade (127.13.224.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } } }
I20260806 13:07:01.089054 24360 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f2fdbe7f433645b982cd7572a06f9ade (127.13.224.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } } }
I20260806 13:07:01.089208 25084 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } } }
I20260806 13:07:01.089272 25084 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:01.089318 25084 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } } }
I20260806 13:07:01.089354 25084 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:01.089406 25084 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } } }
I20260806 13:07:01.089444 25084 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917 [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:01.089649 25097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } } }
I20260806 13:07:01.089752 25097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:07:01.090071 24400 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf238ca059c646dcaa34f2ee20cd1917" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } } }
I20260806 13:07:01.090152 24400 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:07:01.094184 25098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.090390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:40499 (local address 127.0.0.1:54792)
0806 13:07:01.090776 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:01.090784 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:01.090818 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:01.091232 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:01.091233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:01.091235 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:01.091285 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.091286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.092124 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.092125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.093008 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.093011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.093912 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.093913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.093974 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.093978 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:07:01.094069 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 13:07:01.094110 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":78,"threads_started":1}
I20260806 13:07:01.094203 25104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.090599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.13.224.67:40499)
0806 13:07:01.091072 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.091077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.091097 (+ 20us) server_negotiation.cc:407] Connection header received
0806 13:07:01.091135 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.091137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.091153 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.091224 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:01.091335 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.092035 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.093100 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.093830 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.093905 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.093987 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.094086 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.094127 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":75,"threads_started":1}
I20260806 13:07:01.094709 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.090300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:39637 (local address 127.0.0.1:59920)
0806 13:07:01.090351 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:01.090360 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:01.090397 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:01.091093 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:01.091094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:01.091095 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:01.091127 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.091128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.092582 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.092583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.093442 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.093444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.094490 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.094491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.094550 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.094553 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:07:01.094613 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 13:07:01.094652 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":27}
I20260806 13:07:01.094810 25101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.090400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59920 (local address 127.13.224.66:39637)
0806 13:07:01.090918 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.090924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.090933 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:01.090971 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.090973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.090990 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.091203 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:01.091286 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.092496 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.093528 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.094420 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.094481 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.094622 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.094650 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.094696 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":86,"threads_started":1}
I20260806 13:07:01.094949 25100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.090346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:34533 (local address 127.0.0.1:34208)
0806 13:07:01.090587 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:01.090597 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:01.090645 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:01.090948 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:01.090949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:01.090951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:01.091070 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.091073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.092598 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.092599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.093502 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.093506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.094689 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.094690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.094773 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.094777 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:07:01.094842 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 13:07:01.094880 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":78,"threads_started":1}
I20260806 13:07:01.095268 24955 tablet_service.cc:1467] Processing CreateTablet for tablet 114e50a848b543f689cf42f06a22042e (DEFAULT_TABLE table=test-table [id=b1059ea19dde4e2aa4f1c8f6c37629a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:01.095436 24729 tablet_service.cc:1467] Processing CreateTablet for tablet 114e50a848b543f689cf42f06a22042e (DEFAULT_TABLE table=test-table [id=b1059ea19dde4e2aa4f1c8f6c37629a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:01.095696 24955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114e50a848b543f689cf42f06a22042e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:01.095932 24729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114e50a848b543f689cf42f06a22042e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:01.098433 25103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.090588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34208 (local address 127.13.224.65:34533)
0806 13:07:01.090778 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.090783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.090794 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:07:01.090836 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.090838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.090856 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.090938 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:01.091156 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.092512 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.093598 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.094450 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.094516 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.094652 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.098138 (+ 3486us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.098184 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260806 13:07:01.099042 24519 tablet_service.cc:1467] Processing CreateTablet for tablet 114e50a848b543f689cf42f06a22042e (DEFAULT_TABLE table=test-table [id=b1059ea19dde4e2aa4f1c8f6c37629a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:01.099313 24519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114e50a848b543f689cf42f06a22042e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:01.100538 25105 tablet_bootstrap.cc:492] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b: Bootstrap starting.
I20260806 13:07:01.101011 25106 tablet_bootstrap.cc:492] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1: Bootstrap starting.
I20260806 13:07:01.101325 25105 tablet_bootstrap.cc:654] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:01.102089 25106 tablet_bootstrap.cc:654] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:01.102233 25105 log.cc:824] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:01.102799 25106 log.cc:824] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:01.103207 25108 tablet_bootstrap.cc:492] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799: Bootstrap starting.
I20260806 13:07:01.103363 25105 tablet_bootstrap.cc:492] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b: No bootstrap required, opened a new log
I20260806 13:07:01.103432 25105 ts_tablet_manager.cc:1397] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 13:07:01.103890 25106 tablet_bootstrap.cc:492] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1: No bootstrap required, opened a new log
I20260806 13:07:01.103962 25106 ts_tablet_manager.cc:1397] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 13:07:01.104322 25108 tablet_bootstrap.cc:654] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:01.104869 25108 log.cc:824] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:01.105715 25108 tablet_bootstrap.cc:492] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799: No bootstrap required, opened a new log
I20260806 13:07:01.105784 25108 ts_tablet_manager.cc:1397] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 13:07:01.106293 25105 raft_consensus.cc:348] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:01.106442 25105 raft_consensus.cc:374] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:01.106482 25105 raft_consensus.cc:729] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05b2790c44deabe38bccef260ad4b, State: Initialized, Role: FOLLOWER
I20260806 13:07:01.106431 25106 raft_consensus.cc:348] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:01.106591 25106 raft_consensus.cc:374] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:01.106631 25106 raft_consensus.cc:729] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29135061639e45c083246b11b65298a1, State: Initialized, Role: FOLLOWER
I20260806 13:07:01.106617 25105 consensus_queue.cc:260] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:01.106801 25106 consensus_queue.cc:260] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:01.106949 25105 ts_tablet_manager.cc:1428] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:07:01.107077 25060 heartbeater.cc:503] Master 127.13.224.126:35805 was elected leader, sending a full tablet report...
I20260806 13:07:01.107367 25106 ts_tablet_manager.cc:1428] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260806 13:07:01.108390 24834 heartbeater.cc:503] Master 127.13.224.126:35805 was elected leader, sending a full tablet report...
I20260806 13:07:01.109021 25108 raft_consensus.cc:348] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:01.109160 25108 raft_consensus.cc:374] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:01.109196 25108 raft_consensus.cc:729] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6cf2c37a8844c168a3fdfc36a544799, State: Initialized, Role: FOLLOWER
I20260806 13:07:01.109300 25108 consensus_queue.cc:260] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:01.109545 25108 ts_tablet_manager.cc:1428] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260806 13:07:01.109586 24613 heartbeater.cc:503] Master 127.13.224.126:35805 was elected leader, sending a full tablet report...
W20260806 13:07:01.180431 24837 tablet.cc:2369] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 13:07:01.288964 24615 tablet.cc:2369] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 13:07:01.313689 25062 tablet.cc:2369] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:07:01.435222 25112 raft_consensus.cc:482] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:07:01.435349 25112 raft_consensus.cc:504] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:01.435722 25112 leader_election.cc:288] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ec05b2790c44deabe38bccef260ad4b (127.13.224.67:40499), f6cf2c37a8844c168a3fdfc36a544799 (127.13.224.65:34533)
I20260806 13:07:01.438886 25104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.436062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:38771 (local address 127.13.224.67:40499)
0806 13:07:01.436105 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.436106 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.436114 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:07:01.436139 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.436140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.436146 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.436196 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:01.436373 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.436941 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.437910 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.438315 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.438377 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.438387 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.438779 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.438817 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:07:01.438887 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.435895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:40499 (local address 127.13.224.66:38771)
0806 13:07:01.435976 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:01.435989 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:01.436045 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:01.436209 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:01.436211 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:01.436212 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:01.436297 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.436299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.437016 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.437017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.437818 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.437820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.438385 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.438386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.438696 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.438700 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:07:01.438751 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 13:07:01.438795 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 13:07:01.438910 25103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.435984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:51363 (local address 127.13.224.65:34533)
0806 13:07:01.436045 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.436065 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.436090 (+ 25us) server_negotiation.cc:407] Connection header received
0806 13:07:01.436123 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.436124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.436135 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.436182 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:01.436626 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.437163 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.438061 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.438485 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.438560 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.438570 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.438798 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.438846 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 13:07:01.438973 24838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.435950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:34533 (local address 127.13.224.66:51363)
0806 13:07:01.435991 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:01.436002 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:01.436047 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:01.436195 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:01.436196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:01.436197 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:01.436547 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.436550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.437248 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.437249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.437944 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:01.437946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.438568 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.438569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:01.438654 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.438658 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:07:01.438712 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 13:07:01.438753 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 13:07:01.439306 24549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114e50a848b543f689cf42f06a22042e" candidate_uuid: "29135061639e45c083246b11b65298a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cf2c37a8844c168a3fdfc36a544799" is_pre_election: true
I20260806 13:07:01.439345 24986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114e50a848b543f689cf42f06a22042e" candidate_uuid: "29135061639e45c083246b11b65298a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05b2790c44deabe38bccef260ad4b" is_pre_election: true
I20260806 13:07:01.439497 24986 raft_consensus.cc:2454] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29135061639e45c083246b11b65298a1 in term 0.
I20260806 13:07:01.439497 24549 raft_consensus.cc:2454] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29135061639e45c083246b11b65298a1 in term 0.
I20260806 13:07:01.439731 24658 leader_election.cc:302] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29135061639e45c083246b11b65298a1, f6cf2c37a8844c168a3fdfc36a544799; no voters:
I20260806 13:07:01.439874 25112 raft_consensus.cc:2781] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:07:01.439940 25112 raft_consensus.cc:482] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:07:01.439963 25112 raft_consensus.cc:3037] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:01.440642 25112 raft_consensus.cc:504] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:01.440761 25112 leader_election.cc:288] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [CANDIDATE]: Term 1 election: Requested vote from peers 2ec05b2790c44deabe38bccef260ad4b (127.13.224.67:40499), f6cf2c37a8844c168a3fdfc36a544799 (127.13.224.65:34533)
I20260806 13:07:01.440924 24549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114e50a848b543f689cf42f06a22042e" candidate_uuid: "29135061639e45c083246b11b65298a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6cf2c37a8844c168a3fdfc36a544799"
I20260806 13:07:01.440936 24986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114e50a848b543f689cf42f06a22042e" candidate_uuid: "29135061639e45c083246b11b65298a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05b2790c44deabe38bccef260ad4b"
I20260806 13:07:01.441021 24549 raft_consensus.cc:3037] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:01.441021 24986 raft_consensus.cc:3037] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:01.442050 24549 raft_consensus.cc:2454] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29135061639e45c083246b11b65298a1 in term 1.
I20260806 13:07:01.442209 24658 leader_election.cc:302] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29135061639e45c083246b11b65298a1, f6cf2c37a8844c168a3fdfc36a544799; no voters:
I20260806 13:07:01.442312 25112 raft_consensus.cc:2781] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:07:01.442467 25112 raft_consensus.cc:686] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [term 1 LEADER]: Becoming Leader. State: Replica: 29135061639e45c083246b11b65298a1, State: Running, Role: LEADER
I20260806 13:07:01.442570 24986 raft_consensus.cc:2454] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29135061639e45c083246b11b65298a1 in term 1.
I20260806 13:07:01.442663 25112 consensus_queue.cc:237] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:01.443602 23136 catalog_manager.cc:5495] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 29135061639e45c083246b11b65298a1 (127.13.224.66). New cstate: current_term: 1 leader_uuid: "29135061639e45c083246b11b65298a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } health_report { overall_health: UNKNOWN } } }
I20260806 13:07:01.530304 25101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.528043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59922 (local address 127.13.224.66:39637)
0806 13:07:01.528111 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.528113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.528121 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:07:01.528158 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.528158 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.528177 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.528239 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:07:01.528377 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.528851 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.529565 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.529763 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.529820 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.529967 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:07:01.530088 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:07:01.530160 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.530207 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.530227 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 13:07:01.532025 24986 raft_consensus.cc:1261] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b [term 1 FOLLOWER]: Refusing update from remote peer 29135061639e45c083246b11b65298a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:01.532025 24549 raft_consensus.cc:1261] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799 [term 1 FOLLOWER]: Refusing update from remote peer 29135061639e45c083246b11b65298a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:01.532354 25112 consensus_queue.cc:1035] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:01.532505 25114 consensus_queue.cc:1035] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:01.535259 25124 mvcc.cc:205] Tried to move back new op lower bound from 7315544561792602112 to 7315544561429905408. Current Snapshot: MvccSnapshot[applied={T|T < 7315544561792602112}]
I20260806 13:07:01.538679 23137 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:01.538759 23137 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:01.539947 23137 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf238ca059c646dcaa34f2ee20cd1917: Sending DeleteTablet for 3 replicas of tablet 114e50a848b543f689cf42f06a22042e
I20260806 13:07:01.540306 24729 tablet_service.cc:1514] Processing DeleteTablet for tablet 114e50a848b543f689cf42f06a22042e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:01 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:59920
I20260806 13:07:01.540501 24955 tablet_service.cc:1514] Processing DeleteTablet for tablet 114e50a848b543f689cf42f06a22042e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:01 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:54792
I20260806 13:07:01.540722 24519 tablet_service.cc:1514] Processing DeleteTablet for tablet 114e50a848b543f689cf42f06a22042e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:01 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:34208
I20260806 13:07:01.541079 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 23087
I20260806 13:07:01.547045 25137 tablet_replica.cc:330] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799: stopping tablet replica
I20260806 13:07:01.547165 25137 raft_consensus.cc:2229] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:01.547281 25137 raft_consensus.cc:2258] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:01.547442 25134 tablet_replica.cc:330] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1: stopping tablet replica
I20260806 13:07:01.547538 25134 raft_consensus.cc:2229] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:07:01.547628 25134 raft_consensus.cc:2258] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:01.548055 25134 ts_tablet_manager.cc:1905] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:01.548244 25137 ts_tablet_manager.cc:1905] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:01.549719 25137 ts_tablet_manager.cc:1918] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:07:01.549767 25134 ts_tablet_manager.cc:1918] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:07:01.549837 25134 log.cc:1196] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/114e50a848b543f689cf42f06a22042e
I20260806 13:07:01.549934 25137 log.cc:1196] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-0/wal/wals/114e50a848b543f689cf42f06a22042e
I20260806 13:07:01.550129 25134 ts_tablet_manager.cc:1939] T 114e50a848b543f689cf42f06a22042e P 29135061639e45c083246b11b65298a1: Deleting consensus metadata
I20260806 13:07:01.550386 25137 ts_tablet_manager.cc:1939] T 114e50a848b543f689cf42f06a22042e P f6cf2c37a8844c168a3fdfc36a544799: Deleting consensus metadata
W20260806 13:07:01.554569 24446 connection.cc:535] server connection from 127.0.0.1:34208 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 13:07:01.554822 24654 connection.cc:535] server connection from 127.0.0.1:59920 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 13:07:01.555087 25135 tablet_replica.cc:330] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b: stopping tablet replica
I20260806 13:07:01.555197 25135 raft_consensus.cc:2229] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:01.555306 25135 raft_consensus.cc:2258] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:01.556224 25135 ts_tablet_manager.cc:1905] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:01.557901 25135 ts_tablet_manager.cc:1918] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:07:01.557991 25135 log.cc:1196] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/114e50a848b543f689cf42f06a22042e
I20260806 13:07:01.558291 25135 ts_tablet_manager.cc:1939] T 114e50a848b543f689cf42f06a22042e P 2ec05b2790c44deabe38bccef260ad4b: Deleting consensus metadata
W20260806 13:07:01.558820 24878 connection.cc:441] server connection from 127.0.0.1:54792 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54792 (request call id 1) could send its response
I20260806 13:07:01.564038 25075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.556293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49698 (local address 127.13.224.124:42299)
0806 13:07:01.556342 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.556344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.556353 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:01.556447 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.556448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.556463 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.556491 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:01.557251 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.557981 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.558972 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.559212 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.559276 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.560533 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:01.560535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:01.560535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:01.560543 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:01.562257 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.562950 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.562952 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.562952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.563013 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.563217 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.563218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.563219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.563355 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:01.563452 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.563825 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.563942 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:07:01.565770 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:01.556658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38448 (local address 127.13.224.125:34187)
0806 13:07:01.558649 (+ 1991us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:01.558651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:01.558660 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:01.558683 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:01.558684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:01.558697 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:01.558749 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:01.560465 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.561097 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.561765 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:01.561968 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:01.562040 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:01.563176 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:01.563177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:01.563177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:01.563185 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:01.564799 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.565086 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.565087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.565087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.565142 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:01.565307 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:01.565308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:01.565308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:01.565416 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:01.565499 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:01.565594 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 13:07:01.565673 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1967}
I20260806 13:07:02.536345 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:02.536052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.65:44571)
0806 13:07:02.536185 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:02.536291 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
W20260806 13:07:02.536623 24613 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:35805 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
I20260806 13:07:02.537726 25149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:02.537452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.67:44993)
0806 13:07:02.537620 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:02.537682 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":59,"threads_started":1}
W20260806 13:07:02.537937 25060 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:35805 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
I20260806 13:07:02.538074 25151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:02.537507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.66:35397)
0806 13:07:02.537597 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:02.537673 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72,"thread_start_us":43,"threads_started":1}
W20260806 13:07:02.538265 24834 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:35805 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
I20260806 13:07:03.040292 25154 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bf238ca059c646dcaa34f2ee20cd1917)
I20260806 13:07:03.040441 25156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bf238ca059c646dcaa34f2ee20cd1917)
I20260806 13:07:03.040452 25154 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } }
I20260806 13:07:03.040575 25156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } }
I20260806 13:07:03.040918 25154 leader_election.cc:288] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf238ca059c646dcaa34f2ee20cd1917 (127.13.224.126:35805), 225e05dbd8e04293906d6f243481dfd6 (127.13.224.125:34187)
I20260806 13:07:03.041021 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.040814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46708)
0806 13:07:03.040924 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:03.040977 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":51,"threads_started":1}
I20260806 13:07:03.041129 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.040875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46710)
0806 13:07:03.041036 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:03.041091 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":47,"threads_started":1}
W20260806 13:07:03.041132 24326 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
W20260806 13:07:03.041522 24049 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
I20260806 13:07:03.041577 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.041475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46722)
0806 13:07:03.041514 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:03.041550 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 13:07:03.041648 24326 leader_election.cc:334] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bf238ca059c646dcaa34f2ee20cd1917 (127.13.224.126:35805): Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
I20260806 13:07:03.041700 25156 leader_election.cc:288] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf238ca059c646dcaa34f2ee20cd1917 (127.13.224.126:35805), f2fdbe7f433645b982cd7572a06f9ade (127.13.224.124:42299)
I20260806 13:07:03.041949 25161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.041860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46724)
0806 13:07:03.041893 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:03.041922 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260806 13:07:03.042033 24049 leader_election.cc:334] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bf238ca059c646dcaa34f2ee20cd1917 (127.13.224.126:35805): Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
I20260806 13:07:03.044668 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.041035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34187 (local address 127.0.0.1:38460)
0806 13:07:03.041274 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:03.041284 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:03.041338 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:03.041643 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:03.041643 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:03.041647 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:03.041693 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:03.041695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:03.042675 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:03.042676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:03.043922 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:03.043925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:03.044452 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:03.044453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:03.044509 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:03.044515 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 13:07:03.044565 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 13:07:03.044604 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":42,"threads_started":1}
I20260806 13:07:03.044695 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.041408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38460 (local address 127.13.224.125:34187)
0806 13:07:03.041508 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:03.041510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:03.041517 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:03.041538 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:03.041539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:03.041547 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:03.041609 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:03.041776 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:03.042580 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:03.044004 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:03.044382 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:03.044440 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:03.044448 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:03.044580 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 13:07:03.044624 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":52,"threads_started":1}
I20260806 13:07:03.044850 25165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.041284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49706 (local address 127.13.224.124:42299)
0806 13:07:03.041420 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:03.041422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:03.041429 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:03.041449 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:03.041450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:03.041458 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:03.041815 (+ 357us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:03.041843 (+ 28us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:03.042582 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:03.044069 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:03.044465 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:03.044531 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:03.044539 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:03.044731 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 13:07:03.044793 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":88,"threads_started":1}
I20260806 13:07:03.044888 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.041066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42299 (local address 127.0.0.1:49706)
0806 13:07:03.041180 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:03.041226 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:03.041278 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:03.041554 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:03.041555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:03.041557 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:03.041624 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:03.041626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:03.042675 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:03.042676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:03.043922 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:03.043925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:03.044529 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:03.044530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:03.044600 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:03.044604 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:07:03.044649 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 13:07:03.044682 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260806 13:07:03.044999 24086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2fdbe7f433645b982cd7572a06f9ade" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "225e05dbd8e04293906d6f243481dfd6" is_pre_election: true
I20260806 13:07:03.045118 24086 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2fdbe7f433645b982cd7572a06f9ade in term 1.
I20260806 13:07:03.045111 24360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "225e05dbd8e04293906d6f243481dfd6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f2fdbe7f433645b982cd7572a06f9ade" is_pre_election: true
I20260806 13:07:03.045199 24360 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 225e05dbd8e04293906d6f243481dfd6 in term 1.
I20260806 13:07:03.045272 24328 leader_election.cc:302] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 225e05dbd8e04293906d6f243481dfd6, f2fdbe7f433645b982cd7572a06f9ade; no voters: bf238ca059c646dcaa34f2ee20cd1917
I20260806 13:07:03.045328 24045 leader_election.cc:302] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 225e05dbd8e04293906d6f243481dfd6, f2fdbe7f433645b982cd7572a06f9ade; no voters: bf238ca059c646dcaa34f2ee20cd1917
I20260806 13:07:03.045382 25154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 13:07:03.045415 25156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 13:07:03.045435 25154 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 1 FOLLOWER]: Starting leader election (detected failure of leader bf238ca059c646dcaa34f2ee20cd1917)
I20260806 13:07:03.045454 25156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bf238ca059c646dcaa34f2ee20cd1917)
I20260806 13:07:03.045459 25154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 1 FOLLOWER]: Advancing to term 2
I20260806 13:07:03.045483 25156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 1 FOLLOWER]: Advancing to term 2
I20260806 13:07:03.046164 25156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } }
I20260806 13:07:03.046171 25154 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } }
I20260806 13:07:03.046293 25154 leader_election.cc:288] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [CANDIDATE]: Term 2 election: Requested vote from peers bf238ca059c646dcaa34f2ee20cd1917 (127.13.224.126:35805), 225e05dbd8e04293906d6f243481dfd6 (127.13.224.125:34187)
I20260806 13:07:03.046425 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.046342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46736)
0806 13:07:03.046375 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:03.046403 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260806 13:07:03.046490 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.046408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46746)
0806 13:07:03.046445 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:03.046469 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 13:07:03.046432 25156 leader_election.cc:288] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [CANDIDATE]: Term 2 election: Requested vote from peers bf238ca059c646dcaa34f2ee20cd1917 (127.13.224.126:35805), f2fdbe7f433645b982cd7572a06f9ade (127.13.224.124:42299)
I20260806 13:07:03.046563 24360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "225e05dbd8e04293906d6f243481dfd6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f2fdbe7f433645b982cd7572a06f9ade"
I20260806 13:07:03.046628 24360 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 225e05dbd8e04293906d6f243481dfd6 in current term 2: Already voted for candidate f2fdbe7f433645b982cd7572a06f9ade in this term.
I20260806 13:07:03.046670 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.046593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46758)
0806 13:07:03.046630 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:03.046651 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 13:07:03.046728 24049 leader_election.cc:334] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bf238ca059c646dcaa34f2ee20cd1917 (127.13.224.126:35805): Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
I20260806 13:07:03.046437 24086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2fdbe7f433645b982cd7572a06f9ade" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "225e05dbd8e04293906d6f243481dfd6"
I20260806 13:07:03.046801 24045 leader_election.cc:302] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 225e05dbd8e04293906d6f243481dfd6; no voters: bf238ca059c646dcaa34f2ee20cd1917, f2fdbe7f433645b982cd7572a06f9ade
I20260806 13:07:03.046841 24086 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f2fdbe7f433645b982cd7572a06f9ade in current term 2: Already voted for candidate 225e05dbd8e04293906d6f243481dfd6 in this term.
I20260806 13:07:03.046881 25156 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 13:07:03.047094 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.047027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:46764)
0806 13:07:03.047052 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:03.047076 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260806 13:07:03.047168 24326 leader_election.cc:334] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bf238ca059c646dcaa34f2ee20cd1917 (127.13.224.126:35805): Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
I20260806 13:07:03.047217 24326 leader_election.cc:302] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2fdbe7f433645b982cd7572a06f9ade; no voters: 225e05dbd8e04293906d6f243481dfd6, bf238ca059c646dcaa34f2ee20cd1917
I20260806 13:07:03.047298 25154 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 13:07:03.537379 25171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.537051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.65:39805)
0806 13:07:03.537258 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:03.537312 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":129,"threads_started":1}
I20260806 13:07:03.538687 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.538274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.67:36091)
0806 13:07:03.538591 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:03.538642 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":73,"threads_started":1}
I20260806 13:07:03.538816 25173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:03.538562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.66:53619)
0806 13:07:03.538733 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:03.538772 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":58,"threads_started":1}
I20260806 13:07:04.538064 25174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.537776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.65:41003)
0806 13:07:04.537925 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.537977 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260806 13:07:04.539403 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.539184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.67:45975)
0806 13:07:04.539304 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.539357 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260806 13:07:04.539409 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.539210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.66:37567)
0806 13:07:04.539328 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.539364 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
I20260806 13:07:04.834725 25177 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:07:04.834823 25177 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } }
I20260806 13:07:04.834992 25177 leader_election.cc:288] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bf238ca059c646dcaa34f2ee20cd1917 (127.13.224.126:35805), 225e05dbd8e04293906d6f243481dfd6 (127.13.224.125:34187)
I20260806 13:07:04.835249 24086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2fdbe7f433645b982cd7572a06f9ade" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "225e05dbd8e04293906d6f243481dfd6" is_pre_election: true
I20260806 13:07:04.835325 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.835128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:47018)
0806 13:07:04.835250 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.835284 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":52,"threads_started":1}
I20260806 13:07:04.835341 24086 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2fdbe7f433645b982cd7572a06f9ade in term 2.
I20260806 13:07:04.835475 24328 leader_election.cc:302] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 225e05dbd8e04293906d6f243481dfd6, f2fdbe7f433645b982cd7572a06f9ade; no voters:
I20260806 13:07:04.835572 25177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 13:07:04.835609 25177 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:07:04.835628 25177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 2 FOLLOWER]: Advancing to term 3
I20260806 13:07:04.835731 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.835645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:47022)
0806 13:07:04.835686 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.835708 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 13:07:04.835820 24326 leader_election.cc:334] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bf238ca059c646dcaa34f2ee20cd1917 (127.13.224.126:35805): Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
I20260806 13:07:04.836401 25177 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } }
I20260806 13:07:04.836525 25177 leader_election.cc:288] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [CANDIDATE]: Term 3 election: Requested vote from peers bf238ca059c646dcaa34f2ee20cd1917 (127.13.224.126:35805), 225e05dbd8e04293906d6f243481dfd6 (127.13.224.125:34187)
I20260806 13:07:04.836695 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.836600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:47028)
0806 13:07:04.836647 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.836676 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 13:07:04.836678 24086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2fdbe7f433645b982cd7572a06f9ade" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "225e05dbd8e04293906d6f243481dfd6"
I20260806 13:07:04.836755 24086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 2 FOLLOWER]: Advancing to term 3
I20260806 13:07:04.836990 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.836921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:47034)
0806 13:07:04.836955 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.836976 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 13:07:04.837040 24326 leader_election.cc:334] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer bf238ca059c646dcaa34f2ee20cd1917 (127.13.224.126:35805): Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
I20260806 13:07:04.837488 24086 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2fdbe7f433645b982cd7572a06f9ade in term 3.
I20260806 13:07:04.837648 24328 leader_election.cc:302] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 225e05dbd8e04293906d6f243481dfd6, f2fdbe7f433645b982cd7572a06f9ade; no voters: bf238ca059c646dcaa34f2ee20cd1917
I20260806 13:07:04.837723 25177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 3 FOLLOWER]: Leader election won for term 3
I20260806 13:07:04.837841 25177 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [term 3 LEADER]: Becoming Leader. State: Replica: f2fdbe7f433645b982cd7572a06f9ade, State: Running, Role: LEADER
I20260806 13:07:04.837932 25177 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } }
I20260806 13:07:04.838299 25180 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2fdbe7f433645b982cd7572a06f9ade. Latest consensus state: current_term: 3 leader_uuid: "f2fdbe7f433645b982cd7572a06f9ade" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } } }
I20260806 13:07:04.838392 25180 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:04.838582 25183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:07:04.839039 25183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:07:04.839108 25183 catalog_manager.cc:1259] Loaded cluster ID: 757a651f8e0b4b919e896eb6b1b83831
I20260806 13:07:04.839138 25183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:07:04.839331 25183 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:07:04.839404 25183 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: Loaded TSK: 0
I20260806 13:07:04.839546 25183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:07:04.884249 24342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:07:04.885804 24342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:07:04.886821 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.886728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:47058)
0806 13:07:04.886771 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.886798 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 13:07:04.886830 24086 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [term 3 FOLLOWER]: Refusing update from remote peer f2fdbe7f433645b982cd7572a06f9ade: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 13:07:04.887033 25177 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [LEADER]: Connected to new peer: Peer: permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 13:07:04.887131 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.887036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.0.0.1:47064)
0806 13:07:04.887081 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.887107 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 13:07:04.887280 24326 consensus_peers.cc:487] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade -> Peer bf238ca059c646dcaa34f2ee20cd1917 (127.13.224.126:35805): Couldn't send request to peer bf238ca059c646dcaa34f2ee20cd1917. Status: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 13:07:04.887626 25186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2fdbe7f433645b982cd7572a06f9ade. Latest consensus state: current_term: 3 leader_uuid: "f2fdbe7f433645b982cd7572a06f9ade" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } } }
I20260806 13:07:04.887717 25186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:07:04.888128 25180 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f2fdbe7f433645b982cd7572a06f9ade" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } } }
I20260806 13:07:04.888211 25180 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:04.888310 25187 mvcc.cc:205] Tried to move back new op lower bound from 7315544575534714880 to 7315544575336923136. Current Snapshot: MvccSnapshot[applied={T|T < 7315544575534714880}]
I20260806 13:07:04.888280 25186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f2fdbe7f433645b982cd7572a06f9ade" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bf238ca059c646dcaa34f2ee20cd1917" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35805 } } peers { permanent_uuid: "225e05dbd8e04293906d6f243481dfd6" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34187 } attrs { promote: false } } peers { permanent_uuid: "f2fdbe7f433645b982cd7572a06f9ade" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42299 } attrs { promote: false } } }
I20260806 13:07:04.888703 25186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 225e05dbd8e04293906d6f243481dfd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:07:04.892803 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.889803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:39637 (local address 127.0.0.1:59932)
0806 13:07:04.889841 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.889850 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:04.889897 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:04.890238 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:04.890239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:04.890240 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:04.890325 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:04.890327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.891279 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.891280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:04.892004 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:04.892006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.892600 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.892601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:04.892663 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:04.892668 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:07:04.892696 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 13:07:04.892736 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":17}
I20260806 13:07:04.893047 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.890000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59932 (local address 127.13.224.66:39637)
0806 13:07:04.890115 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:04.890117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:04.890124 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:04.890146 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:04.890146 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:04.890156 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:04.890225 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:04.890413 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.891177 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.892082 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.892511 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.892905 (+ 394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:04.892919 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:04.892941 (+ 22us) server_negotiation.cc:299] Negotiation successful
0806 13:07:04.892980 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260806 13:07:04.893301 24729 tablet_service.cc:1467] Processing CreateTablet for tablet 37069a26249c43e39e499fef8c382d72 (DEFAULT_TABLE table=test-table [id=da3140994ccb49468805391b440435ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:04.893370 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.889865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:34533 (local address 127.0.0.1:34216)
0806 13:07:04.890314 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.890324 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:04.890371 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:04.890563 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:04.890564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:04.890566 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:04.890616 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:04.890618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.891234 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.891235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:04.892411 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:04.892414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.893144 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.893145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:04.893209 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:04.893214 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:07:04.893261 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 13:07:04.893302 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":24,"thread_start_us":72,"threads_started":1}
I20260806 13:07:04.893437 24729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37069a26249c43e39e499fef8c382d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:04.893582 25197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.890046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34216 (local address 127.13.224.65:34533)
0806 13:07:04.890152 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:04.890154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:04.890420 (+ 266us) server_negotiation.cc:407] Connection header received
0806 13:07:04.890452 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:04.890453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:04.890463 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:04.890516 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:04.890684 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.891161 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.892500 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.893060 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.893122 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:04.893135 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:04.893462 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 13:07:04.893514 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":61,"threads_started":1}
I20260806 13:07:04.893770 24519 tablet_service.cc:1467] Processing CreateTablet for tablet 37069a26249c43e39e499fef8c382d72 (DEFAULT_TABLE table=test-table [id=da3140994ccb49468805391b440435ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:04.893886 24519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37069a26249c43e39e499fef8c382d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:04.894009 25193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.889821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:40499 (local address 127.0.0.1:54804)
0806 13:07:04.889946 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.889955 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:04.890001 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:04.890235 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:04.890236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:04.890239 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:04.890325 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:04.890327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.891285 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.891285 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:04.892447 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:04.892448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.893787 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.893788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:04.893845 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:04.893849 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:07:04.893909 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 13:07:04.893946 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":58,"thread_start_us":85,"threads_started":1}
I20260806 13:07:04.894057 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.889978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54804 (local address 127.13.224.67:40499)
0806 13:07:04.890108 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:04.890110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:04.890118 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:07:04.890145 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:04.890146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:04.890156 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:04.890229 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:04.890412 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.891177 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.893171 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.893669 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.893778 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:04.893805 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:04.893919 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 13:07:04.893959 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260806 13:07:04.894241 24955 tablet_service.cc:1467] Processing CreateTablet for tablet 37069a26249c43e39e499fef8c382d72 (DEFAULT_TABLE table=test-table [id=da3140994ccb49468805391b440435ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:04.894344 24955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37069a26249c43e39e499fef8c382d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:04.895184 25200 tablet_bootstrap.cc:492] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799: Bootstrap starting.
I20260806 13:07:04.895478 25200 tablet_bootstrap.cc:654] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:04.895625 25202 tablet_bootstrap.cc:492] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b: Bootstrap starting.
I20260806 13:07:04.895720 25198 tablet_bootstrap.cc:492] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1: Bootstrap starting.
I20260806 13:07:04.896014 25202 tablet_bootstrap.cc:654] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:04.896113 25198 tablet_bootstrap.cc:654] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:04.896119 25200 tablet_bootstrap.cc:492] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799: No bootstrap required, opened a new log
I20260806 13:07:04.896167 25200 ts_tablet_manager.cc:1397] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:04.896375 25200 raft_consensus.cc:348] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:04.896450 25200 raft_consensus.cc:374] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:04.896500 25200 raft_consensus.cc:729] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6cf2c37a8844c168a3fdfc36a544799, State: Initialized, Role: FOLLOWER
I20260806 13:07:04.896576 25202 tablet_bootstrap.cc:492] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b: No bootstrap required, opened a new log
I20260806 13:07:04.896579 25200 consensus_queue.cc:260] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:04.896631 25202 ts_tablet_manager.cc:1397] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:04.896649 25198 tablet_bootstrap.cc:492] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1: No bootstrap required, opened a new log
I20260806 13:07:04.896692 25198 ts_tablet_manager.cc:1397] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:04.896768 25200 ts_tablet_manager.cc:1428] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 13:07:04.896771 25202 raft_consensus.cc:348] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:04.896834 25202 raft_consensus.cc:374] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:04.896852 25202 raft_consensus.cc:729] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05b2790c44deabe38bccef260ad4b, State: Initialized, Role: FOLLOWER
I20260806 13:07:04.896889 25202 consensus_queue.cc:260] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:04.896870 25198 raft_consensus.cc:348] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:04.896936 25198 raft_consensus.cc:374] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:04.896956 25198 raft_consensus.cc:729] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29135061639e45c083246b11b65298a1, State: Initialized, Role: FOLLOWER
I20260806 13:07:04.897003 25174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.896907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.65:52907)
0806 13:07:04.896950 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.896975 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 13:07:04.897011 25198 consensus_queue.cc:260] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:04.897069 25202 ts_tablet_manager.cc:1428] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:04.897172 25198 ts_tablet_manager.cc:1428] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:04.897496 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.897356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.66:39239)
0806 13:07:04.897444 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.897469 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 13:07:04.897689 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.897423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.67:40641)
0806 13:07:04.897618 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.897664 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260806 13:07:04.897985 24830 heartbeater.cc:503] Master 127.13.224.124:42299 was elected leader, sending a full tablet report...
I20260806 13:07:04.898257 24609 heartbeater.cc:503] Master 127.13.224.124:42299 was elected leader, sending a full tablet report...
I20260806 13:07:04.898512 25058 heartbeater.cc:503] Master 127.13.224.124:42299 was elected leader, sending a full tablet report...
I20260806 13:07:04.898615 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.898512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.66:39529)
0806 13:07:04.898559 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.898590 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 13:07:04.898908 25174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.898180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.65:58805)
0806 13:07:04.898229 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.898252 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 13:07:04.899219 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.899118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.67:49043)
0806 13:07:04.899168 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.899196 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 13:07:04.936267 25209 raft_consensus.cc:482] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:07:04.936347 25209 raft_consensus.cc:504] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:04.936630 25209 leader_election.cc:288] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29135061639e45c083246b11b65298a1 (127.13.224.66:39637), 2ec05b2790c44deabe38bccef260ad4b (127.13.224.67:40499)
I20260806 13:07:04.940299 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.936837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:50087 (local address 127.13.224.67:40499)
0806 13:07:04.936909 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:04.936911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:04.936999 (+ 88us) server_negotiation.cc:407] Connection header received
0806 13:07:04.937042 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:04.937043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:04.937051 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:04.937102 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:04.937210 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.937928 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.939411 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.939955 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.940024 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:04.940034 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:04.940193 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 13:07:04.940231 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 13:07:04.940299 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.936860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:44003 (local address 127.13.224.66:39637)
0806 13:07:04.936903 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:04.936905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:04.936914 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:04.936939 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:04.936939 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:04.936949 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:04.937007 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:04.937164 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.937922 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.939411 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.939956 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.940024 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:04.940034 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:04.940193 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 13:07:04.940231 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:07:04.940335 25174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.936764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:39637 (local address 127.13.224.65:44003)
0806 13:07:04.936805 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.936817 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:04.936870 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:04.937020 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:04.937021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:04.937024 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:04.937103 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:04.937105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.938036 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.938037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:04.939330 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:04.939333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.940056 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.940057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:04.940126 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:04.940132 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 13:07:04.940182 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 13:07:04.940231 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":64}
I20260806 13:07:04.940354 25214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.936782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:40499 (local address 127.13.224.65:50087)
0806 13:07:04.936940 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.936947 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:04.936994 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:04.937109 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:04.937110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:04.937111 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:04.937155 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:04.937156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.937993 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.937993 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:04.939331 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:04.939333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.940039 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.940040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:04.940126 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:04.940132 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 13:07:04.940182 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 13:07:04.940231 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":57,"thread_start_us":100,"threads_started":1}
I20260806 13:07:04.940580 24986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37069a26249c43e39e499fef8c382d72" candidate_uuid: "f6cf2c37a8844c168a3fdfc36a544799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05b2790c44deabe38bccef260ad4b" is_pre_election: true
I20260806 13:07:04.940662 24986 raft_consensus.cc:2454] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6cf2c37a8844c168a3fdfc36a544799 in term 0.
I20260806 13:07:04.940728 24759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37069a26249c43e39e499fef8c382d72" candidate_uuid: "f6cf2c37a8844c168a3fdfc36a544799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29135061639e45c083246b11b65298a1" is_pre_election: true
I20260806 13:07:04.940833 24443 leader_election.cc:302] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05b2790c44deabe38bccef260ad4b, f6cf2c37a8844c168a3fdfc36a544799; no voters:
I20260806 13:07:04.940871 24759 raft_consensus.cc:2454] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6cf2c37a8844c168a3fdfc36a544799 in term 0.
I20260806 13:07:04.941021 25209 raft_consensus.cc:2781] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:07:04.941074 25209 raft_consensus.cc:482] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:07:04.941095 25209 raft_consensus.cc:3037] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:04.941763 25209 raft_consensus.cc:504] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:04.941892 25209 leader_election.cc:288] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [CANDIDATE]: Term 1 election: Requested vote from peers 29135061639e45c083246b11b65298a1 (127.13.224.66:39637), 2ec05b2790c44deabe38bccef260ad4b (127.13.224.67:40499)
I20260806 13:07:04.942052 24759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37069a26249c43e39e499fef8c382d72" candidate_uuid: "f6cf2c37a8844c168a3fdfc36a544799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29135061639e45c083246b11b65298a1"
I20260806 13:07:04.942060 24986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37069a26249c43e39e499fef8c382d72" candidate_uuid: "f6cf2c37a8844c168a3fdfc36a544799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05b2790c44deabe38bccef260ad4b"
I20260806 13:07:04.942123 24986 raft_consensus.cc:3037] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:04.942124 24759 raft_consensus.cc:3037] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:04.942673 24986 raft_consensus.cc:2454] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6cf2c37a8844c168a3fdfc36a544799 in term 1.
I20260806 13:07:04.942811 24759 raft_consensus.cc:2454] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6cf2c37a8844c168a3fdfc36a544799 in term 1.
I20260806 13:07:04.942830 24443 leader_election.cc:302] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05b2790c44deabe38bccef260ad4b, f6cf2c37a8844c168a3fdfc36a544799; no voters:
I20260806 13:07:04.942962 25209 raft_consensus.cc:2781] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:07:04.943130 25209 raft_consensus.cc:686] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [term 1 LEADER]: Becoming Leader. State: Replica: f6cf2c37a8844c168a3fdfc36a544799, State: Running, Role: LEADER
I20260806 13:07:04.943195 25209 consensus_queue.cc:237] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } }
I20260806 13:07:04.943475 25214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.943378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.65:51049)
0806 13:07:04.943411 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.943443 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260806 13:07:04.943959 24342 catalog_manager.cc:5495] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6cf2c37a8844c168a3fdfc36a544799 (127.13.224.65). New cstate: current_term: 1 leader_uuid: "f6cf2c37a8844c168a3fdfc36a544799" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6cf2c37a8844c168a3fdfc36a544799" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 34533 } health_report { overall_health: HEALTHY } } }
I20260806 13:07:04.949970 25197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.948089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34224 (local address 127.13.224.65:34533)
0806 13:07:04.948148 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:04.948150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:04.948157 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:04.948205 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:04.948205 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:04.948220 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:04.948280 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:07:04.948378 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.948816 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.949456 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:04.949621 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:04.949661 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:04.949752 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:07:04.949840 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:07:04.949885 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:04.949901 (+ 16us) server_negotiation.cc:299] Negotiation successful
0806 13:07:04.949916 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 13:07:04.950881 24986 raft_consensus.cc:1261] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b [term 1 FOLLOWER]: Refusing update from remote peer f6cf2c37a8844c168a3fdfc36a544799: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:04.950951 24759 raft_consensus.cc:1261] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1 [term 1 FOLLOWER]: Refusing update from remote peer f6cf2c37a8844c168a3fdfc36a544799: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:04.951105 25209 consensus_queue.cc:1035] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05b2790c44deabe38bccef260ad4b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 40499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:04.951218 25217 consensus_queue.cc:1035] T 37069a26249c43e39e499fef8c382d72 P f6cf2c37a8844c168a3fdfc36a544799 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29135061639e45c083246b11b65298a1" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:04.951934 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.951835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.66:45275)
0806 13:07:04.951879 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.951904 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 13:07:04.952082 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.951994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.67:60143)
0806 13:07:04.952035 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.952058 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 13:07:04.953351 25223 mvcc.cc:205] Tried to move back new op lower bound from 7315544575797338112 to 7315544575768186880. Current Snapshot: MvccSnapshot[applied={T|T < 7315544575797338112}]
I20260806 13:07:04.953426 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.953349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.66:36231)
0806 13:07:04.953383 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.953406 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 13:07:04.953622 25214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.953309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.65:33031)
0806 13:07:04.953566 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.953596 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240}
I20260806 13:07:04.953703 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.953621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35805 (local address 127.13.224.67:47371)
0806 13:07:04.953653 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.953678 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 13:07:04.954406 25222 mvcc.cc:205] Tried to move back new op lower bound from 7315544575797338112 to 7315544575768186880. Current Snapshot: MvccSnapshot[applied={T|T < 7315544575797338112}]
I20260806 13:07:04.955658 24342 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:04.955746 24342 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:04.956843 24342 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f2fdbe7f433645b982cd7572a06f9ade: Sending DeleteTablet for 3 replicas of tablet 37069a26249c43e39e499fef8c382d72
I20260806 13:07:04.957134 24519 tablet_service.cc:1514] Processing DeleteTablet for tablet 37069a26249c43e39e499fef8c382d72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:04 UTC) from {username='oryx', principal='oryx/127.13.224.124@KRBTEST.COM'} at 127.0.0.1:34216
I20260806 13:07:04.957172 24955 tablet_service.cc:1514] Processing DeleteTablet for tablet 37069a26249c43e39e499fef8c382d72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:04 UTC) from {username='oryx', principal='oryx/127.13.224.124@KRBTEST.COM'} at 127.0.0.1:54804
I20260806 13:07:04.957376 25236 tablet_replica.cc:330] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b: stopping tablet replica
I20260806 13:07:04.957361 24729 tablet_service.cc:1514] Processing DeleteTablet for tablet 37069a26249c43e39e499fef8c382d72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:04 UTC) from {username='oryx', principal='oryx/127.13.224.124@KRBTEST.COM'} at 127.0.0.1:59932
I20260806 13:07:04.957432 25236 raft_consensus.cc:2229] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:04.957479 25236 raft_consensus.cc:2258] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:04.957551 25238 tablet_replica.cc:330] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1: stopping tablet replica
I20260806 13:07:04.957621 25238 raft_consensus.cc:2229] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:04.957677 25238 raft_consensus.cc:2258] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:04.957773 25236 ts_tablet_manager.cc:1905] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:04.957820 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 24418
I20260806 13:07:04.958086 25238 ts_tablet_manager.cc:1905] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:04.959607 25238 ts_tablet_manager.cc:1918] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:07:04.959659 25238 log.cc:1196] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/37069a26249c43e39e499fef8c382d72
I20260806 13:07:04.959764 25236 ts_tablet_manager.cc:1918] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:07:04.959836 25236 log.cc:1196] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/37069a26249c43e39e499fef8c382d72
I20260806 13:07:04.959894 25238 ts_tablet_manager.cc:1939] T 37069a26249c43e39e499fef8c382d72 P 29135061639e45c083246b11b65298a1: Deleting consensus metadata
I20260806 13:07:04.960107 25236 ts_tablet_manager.cc:1939] T 37069a26249c43e39e499fef8c382d72 P 2ec05b2790c44deabe38bccef260ad4b: Deleting consensus metadata
I20260806 13:07:04.960187 24325 catalog_manager.cc:4841] TS 29135061639e45c083246b11b65298a1 (127.13.224.66:39637): tablet 37069a26249c43e39e499fef8c382d72 (table test-table [id=da3140994ccb49468805391b440435ad]) successfully deleted
I20260806 13:07:04.960420 24323 catalog_manager.cc:4841] TS 2ec05b2790c44deabe38bccef260ad4b (127.13.224.67:40499): tablet 37069a26249c43e39e499fef8c382d72 (table test-table [id=da3140994ccb49468805391b440435ad]) successfully deleted
I20260806 13:07:04.970129 25193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:04.969991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:34533 (local address 127.0.0.1:34226)
0806 13:07:04.970065 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:04.970093 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:34533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 13:07:04.970280 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 24623
W20260806 13:07:04.970294 24328 catalog_manager.cc:4584] TS f6cf2c37a8844c168a3fdfc36a544799 (127.13.224.65:34533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37069a26249c43e39e499fef8c382d72: Network error: Client connection negotiation failed: client connection to 127.13.224.65:34533: connect: Connection refused (error 111)
I20260806 13:07:04.980816 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 24844
I20260806 13:07:04.989346 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 23856
I20260806 13:07:05.003296 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 24157
2026-08-06T13:07:05Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7254 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:07:05 dist-test-slave-q9h9 krb5kdc[25245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:07:05 dist-test-slave-q9h9 krb5kdc[25245](info): set up 2 sockets
Aug 06 13:07:05 dist-test-slave-q9h9 krb5kdc[25245](info): commencing operation
krb5kdc: starting...
W20260806 13:07:06.118938 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:07:06 dist-test-slave-q9h9 krb5kdc[25245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:07:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:07:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.126.keytab.
Entry for principal oryx/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.126.keytab.
I20260806 13:07:06.230899 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35293
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.126.keytab
--principal=oryx/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:35293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:06.311059 25262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:06.311254 25262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:06.311291 25262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:06.312431 25262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:07:06.312470 25262 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 13:07:06.312487 25262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:06.312505 25262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:06.312518 25262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:07:06.312534 25262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:07:06.314050 25262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:35293
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.126.keytab
--principal=oryx/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:06.314293 25262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:06.314504 25262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:06.319094 25274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:06.319202 25262 server_base.cc:1034] running on GCE node
W20260806 13:07:06.319110 25271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:06.319114 25272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:06.319592 25262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:06.319881 25262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:06.321024 25262 hybrid_clock.cc:648] HybridClock initialized: now 1786021626321001 us; error 40 us; skew 500 ppm
Aug 06 13:07:06 dist-test-slave-q9h9 krb5kdc[25245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021626, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:06.323424 25262 init.cc:377] Logged in from keytab as oryx/127.13.224.126@KRBTEST.COM (short username oryx)
I20260806 13:07:06.323869 25262 webserver.cc:460] Webserver started at http://127.13.224.126:45273/ using document root <none> and password file <none>
I20260806 13:07:06.324080 25262 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:06.324128 25262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:06.324250 25262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:06.325245 25262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "d91c7e3dd03c49a7b11dd64c17fb0cf8"
format_stamp: "Formatted at 2026-08-06 13:07:06 on dist-test-slave-q9h9"
server_key: "e3d734fe28d3c22f28eb2d1c04deb8fd"
server_key_iv: "376c3700ba8b1738745bfeb43e53b8c2"
server_key_version: "encryptionkey@0"
I20260806 13:07:06.325567 25262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "d91c7e3dd03c49a7b11dd64c17fb0cf8"
format_stamp: "Formatted at 2026-08-06 13:07:06 on dist-test-slave-q9h9"
server_key: "e3d734fe28d3c22f28eb2d1c04deb8fd"
server_key_iv: "376c3700ba8b1738745bfeb43e53b8c2"
server_key_version: "encryptionkey@0"
I20260806 13:07:06.326884 25262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:07:06.327562 25281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:06.327720 25262 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:06.327821 25262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "d91c7e3dd03c49a7b11dd64c17fb0cf8"
format_stamp: "Formatted at 2026-08-06 13:07:06 on dist-test-slave-q9h9"
server_key: "e3d734fe28d3c22f28eb2d1c04deb8fd"
server_key_iv: "376c3700ba8b1738745bfeb43e53b8c2"
server_key_version: "encryptionkey@0"
I20260806 13:07:06.327883 25262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:06.343016 25262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:06.345068 25262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:06.345208 25262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:06.349536 25262 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:35293
I20260806 13:07:06.349576 25364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:35293 every 8 connection(s)
I20260806 13:07:06.349892 25262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:07:06.350549 25365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:06.352790 25365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8: Bootstrap starting.
I20260806 13:07:06.353366 25365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:06.353634 25365 log.cc:824] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:06.354300 25365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8: No bootstrap required, opened a new log
I20260806 13:07:06.355022 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 25262
I20260806 13:07:06.355126 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:07:06.355321 14209 external_mini_cluster.cc:1428] Setting key c9fd1ed402f9e80502c107362ef492d7
I20260806 13:07:06.356714 25365 raft_consensus.cc:348] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d91c7e3dd03c49a7b11dd64c17fb0cf8" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35293 } }
I20260806 13:07:06.356848 25365 raft_consensus.cc:374] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:06.356884 25365 raft_consensus.cc:729] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d91c7e3dd03c49a7b11dd64c17fb0cf8, State: Initialized, Role: FOLLOWER
I20260806 13:07:06.357061 25365 consensus_queue.cc:260] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d91c7e3dd03c49a7b11dd64c17fb0cf8" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35293 } }
I20260806 13:07:06.357146 25365 raft_consensus.cc:388] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:07:06.357218 25365 raft_consensus.cc:482] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:07:06.357270 25365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:06.357959 25365 raft_consensus.cc:504] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d91c7e3dd03c49a7b11dd64c17fb0cf8" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35293 } }
I20260806 13:07:06.358078 25365 leader_election.cc:302] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d91c7e3dd03c49a7b11dd64c17fb0cf8; no voters:
I20260806 13:07:06.358242 25365 leader_election.cc:288] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:07:06.358284 25372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:07:06.358459 25372 raft_consensus.cc:686] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [term 1 LEADER]: Becoming Leader. State: Replica: d91c7e3dd03c49a7b11dd64c17fb0cf8, State: Running, Role: LEADER
I20260806 13:07:06.358484 25365 sys_catalog.cc:564] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:07:06.358649 25372 consensus_queue.cc:237] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d91c7e3dd03c49a7b11dd64c17fb0cf8" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35293 } }
I20260806 13:07:06.359198 25374 sys_catalog.cc:455] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d91c7e3dd03c49a7b11dd64c17fb0cf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d91c7e3dd03c49a7b11dd64c17fb0cf8" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35293 } } }
I20260806 13:07:06.359436 25374 sys_catalog.cc:458] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:06.359369 25375 sys_catalog.cc:455] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d91c7e3dd03c49a7b11dd64c17fb0cf8. Latest consensus state: current_term: 1 leader_uuid: "d91c7e3dd03c49a7b11dd64c17fb0cf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d91c7e3dd03c49a7b11dd64c17fb0cf8" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35293 } } }
I20260806 13:07:06.359907 25390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:07:06.359935 25375 sys_catalog.cc:458] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8 [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:06.360411 25390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:07:06.361949 25390 catalog_manager.cc:1347] Generated new cluster ID: 8a8adb0cd8fa407b9cacea16d59909e4
I20260806 13:07:06.362003 25390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:07:06 dist-test-slave-q9h9 krb5kdc[25245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
I20260806 13:07:06.366180 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.355831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46874 (local address 127.13.224.126:35293)
0806 13:07:06.355984 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.355991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.356016 (+ 25us) server_negotiation.cc:407] Connection header received
0806 13:07:06.356686 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.356693 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.356742 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.356871 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:06.357652 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.358979 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.360040 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.360580 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.360683 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.363238 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:06.363242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:06.363243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:06.363254 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:06.364718 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.365042 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.365045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.365045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.365139 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.365323 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.365323 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.365324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.365546 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:06.365694 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.365798 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.365931 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260806 13:07:06.383088 25390 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:07:06.383723 25390 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:07:06.387738 25390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8: Generated new TSK 0
I20260806 13:07:06.387930 25390 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab.
Entry for principal oryx/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab.
I20260806 13:07:06.420390 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35293
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab
--principal=oryx/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:06.524000 25400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:06.524261 25400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:06.524287 25400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:06.525388 25400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:06.525424 25400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:06.525444 25400 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 13:07:06.525462 25400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:07:06.526993 25400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab
--principal=oryx/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.224.126:35293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:06.527194 25400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:06.527381 25400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:06.532235 25408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:06.532240 25405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:06.532235 25406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:06.532462 25400 server_base.cc:1034] running on GCE node
I20260806 13:07:06.532718 25400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:06.532971 25400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:06.534133 25400 hybrid_clock.cc:648] HybridClock initialized: now 1786021626534116 us; error 33 us; skew 500 ppm
Aug 06 13:07:06 dist-test-slave-q9h9 krb5kdc[25245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021626, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:06.536557 25400 init.cc:377] Logged in from keytab as oryx/127.13.224.65@KRBTEST.COM (short username oryx)
I20260806 13:07:06.536976 25400 webserver.cc:460] Webserver started at http://127.13.224.65:44645/ using document root <none> and password file <none>
I20260806 13:07:06.537201 25400 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:06.537253 25400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:06.537389 25400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:06.538475 25400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "591f2c06adac47ae96d94bf1ed013bf6"
format_stamp: "Formatted at 2026-08-06 13:07:06 on dist-test-slave-q9h9"
server_key: "88df3e1d18b61e76de449a5509a365e1"
server_key_iv: "6de9ef158d61587645650735eb7a0b0a"
server_key_version: "encryptionkey@0"
I20260806 13:07:06.538822 25400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "591f2c06adac47ae96d94bf1ed013bf6"
format_stamp: "Formatted at 2026-08-06 13:07:06 on dist-test-slave-q9h9"
server_key: "88df3e1d18b61e76de449a5509a365e1"
server_key_iv: "6de9ef158d61587645650735eb7a0b0a"
server_key_version: "encryptionkey@0"
I20260806 13:07:06.540184 25400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:07:06.540859 25417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:06.541100 25400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:06.541181 25400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "591f2c06adac47ae96d94bf1ed013bf6"
format_stamp: "Formatted at 2026-08-06 13:07:06 on dist-test-slave-q9h9"
server_key: "88df3e1d18b61e76de449a5509a365e1"
server_key_iv: "6de9ef158d61587645650735eb7a0b0a"
server_key_version: "encryptionkey@0"
I20260806 13:07:06.541263 25400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:06.560869 25400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:06.562815 25400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:06.562947 25400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:06.563469 25400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:06.563508 25400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:06.563537 25400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:06.563560 25400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 13:07:06 dist-test-slave-q9h9 krb5kdc[25245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021626, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
I20260806 13:07:06.570318 25400 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:38237
I20260806 13:07:06.570393 25591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:38237 every 8 connection(s)
I20260806 13:07:06.570672 25400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:07:06.572398 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.563817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:55129 (local address 127.13.224.126:35293)
0806 13:07:06.563882 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.563884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.564041 (+ 157us) server_negotiation.cc:407] Connection header received
0806 13:07:06.564494 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.564496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.564508 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.564587 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:06.565185 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.565839 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.566881 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.567090 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.567148 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.569942 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:06.569943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:06.569943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:06.569950 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:06.571377 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.571722 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.571722 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.571723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.571777 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.571949 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.571949 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.571950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.572041 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:06.572083 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.572270 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.572335 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 13:07:06.572561 25436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.563779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35293 (local address 127.13.224.65:55129)
0806 13:07:06.563918 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:06.563958 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:06.564407 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:06.564636 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:06.564639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:06.564870 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:06.565096 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.565101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.565949 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.565950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.566797 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.566800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.567169 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.567170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.567295 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.567758 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:06.567769 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:06.569849 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:06.571445 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:06.571448 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:06.571449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:06.571649 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:06.571825 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:06.571826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:06.571826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:06.571913 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:06.572081 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:06.572083 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:06.572185 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 13:07:06.572258 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 13:07:06.572369 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260806 13:07:06.573014 25592 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35293
I20260806 13:07:06.573091 25592 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:06.573269 25592 heartbeater.cc:511] Master 127.13.224.126:35293 requested a full tablet report, sending...
I20260806 13:07:06.573787 25302 ts_manager.cc:194] Registered new tserver with Master: 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237)
I20260806 13:07:06.574455 25302 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:55129
I20260806 13:07:06.576089 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 25400
I20260806 13:07:06.576176 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:07:06.576375 14209 external_mini_cluster.cc:1428] Setting key a2f51437329c345cf46eb07f23894fcb
I20260806 13:07:06.579090 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.573974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:37361 (local address 127.13.224.126:35293)
0806 13:07:06.574007 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.574008 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.574137 (+ 129us) server_negotiation.cc:407] Connection header received
0806 13:07:06.574192 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.574193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.574202 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.574246 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:06.574561 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.575033 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.575717 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.575902 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.575976 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.576840 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:06.576841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:06.576841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:06.576848 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:06.578072 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.578427 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.578428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.578428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.578488 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.578667 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.578668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.578668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.578797 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:06.578850 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.578983 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.579037 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 13:07:06.579154 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.573929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35293 (local address 127.13.224.65:37361)
0806 13:07:06.574077 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:06.574086 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:06.574141 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:06.574250 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:06.574251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:06.574422 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:06.574467 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.574468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.575123 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.575124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.575640 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.575642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.575983 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.575984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.576041 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.576426 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:06.576431 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:06.576774 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:06.578140 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:06.578140 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:06.578141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:06.578313 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:06.578542 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:06.578542 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:06.578543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:06.578607 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:06.578851 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:06.578852 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:06.578899 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 13:07:06.578967 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 13:07:06.579058 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for oryx/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab.
Entry for principal oryx/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab.
I20260806 13:07:06.599766 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35293
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab
--principal=oryx/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:06.679955 25610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:06.680143 25610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:06.680166 25610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:06.681231 25610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:06.681268 25610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:06.681288 25610 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 13:07:06.681303 25610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:07:06.682782 25610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab
--principal=oryx/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.224.126:35293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:06.682955 25610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:06.683115 25610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:06.687721 25617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:06.687758 25621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:06.687795 25618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:06.687875 25610 server_base.cc:1034] running on GCE node
I20260806 13:07:06.688127 25610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:06.688328 25610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:06.689457 25610 hybrid_clock.cc:648] HybridClock initialized: now 1786021626689443 us; error 30 us; skew 500 ppm
Aug 06 13:07:06 dist-test-slave-q9h9 krb5kdc[25245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021626, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:06.691816 25610 init.cc:377] Logged in from keytab as oryx/127.13.224.66@KRBTEST.COM (short username oryx)
I20260806 13:07:06.692206 25610 webserver.cc:460] Webserver started at http://127.13.224.66:35025/ using document root <none> and password file <none>
I20260806 13:07:06.692395 25610 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:06.692438 25610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:06.692556 25610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:06.693532 25610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "dba5659cf28047c090f6073d5779851b"
format_stamp: "Formatted at 2026-08-06 13:07:06 on dist-test-slave-q9h9"
server_key: "70170c3138101078092916fb9b3568eb"
server_key_iv: "232209061fe1d9e324aaae76c2587f0a"
server_key_version: "encryptionkey@0"
I20260806 13:07:06.693890 25610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "dba5659cf28047c090f6073d5779851b"
format_stamp: "Formatted at 2026-08-06 13:07:06 on dist-test-slave-q9h9"
server_key: "70170c3138101078092916fb9b3568eb"
server_key_iv: "232209061fe1d9e324aaae76c2587f0a"
server_key_version: "encryptionkey@0"
I20260806 13:07:06.695148 25610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:07:06.695778 25630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:06.695931 25610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:06.696012 25610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "dba5659cf28047c090f6073d5779851b"
format_stamp: "Formatted at 2026-08-06 13:07:06 on dist-test-slave-q9h9"
server_key: "70170c3138101078092916fb9b3568eb"
server_key_iv: "232209061fe1d9e324aaae76c2587f0a"
server_key_version: "encryptionkey@0"
I20260806 13:07:06.696064 25610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:06.709960 25610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:06.711926 25610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:06.712033 25610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:06.712456 25610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:06.712493 25610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:06.712512 25610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:06.712523 25610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:06.718626 25610 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:39475
I20260806 13:07:06.718688 25789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:39475 every 8 connection(s)
Aug 06 13:07:06 dist-test-slave-q9h9 krb5kdc[25245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021626, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
I20260806 13:07:06.719012 25610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:07:06.721632 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.712883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:42489 (local address 127.13.224.126:35293)
0806 13:07:06.712947 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.712949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.713144 (+ 195us) server_negotiation.cc:407] Connection header received
0806 13:07:06.713652 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.713653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.713667 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.713745 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:06.714583 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.715333 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.716777 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.717050 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.717125 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.719423 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:06.719424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:06.719425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:06.719431 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:06.720668 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.720987 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.720987 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.720988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.721032 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.721187 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.721187 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.721187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.721288 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:06.721332 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.721517 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.721571 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 13:07:06.721807 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.712844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35293 (local address 127.13.224.66:42489)
0806 13:07:06.713022 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:06.713062 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:06.713534 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:06.713823 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:06.713827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:06.714214 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:06.714482 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.714489 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.715433 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.715434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.716670 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.716674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.717129 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.717131 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.717228 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.717658 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:06.717669 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:06.719341 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:06.720730 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:06.720735 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:06.720737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:06.720931 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:06.721069 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:06.721070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:06.721070 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:06.721148 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:06.721331 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:06.721334 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:06.721432 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 13:07:06.721504 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 13:07:06.721612 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":66,"threads_started":1}
I20260806 13:07:06.722215 25790 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35293
I20260806 13:07:06.722311 25790 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:06.722472 25790 heartbeater.cc:511] Master 127.13.224.126:35293 requested a full tablet report, sending...
I20260806 13:07:06.722869 25301 ts_manager.cc:194] Registered new tserver with Master: dba5659cf28047c090f6073d5779851b (127.13.224.66:39475)
I20260806 13:07:06.723326 25301 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:42489
I20260806 13:07:06.724064 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 25610
I20260806 13:07:06.724174 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:07:06.724349 14209 external_mini_cluster.cc:1428] Setting key 5a3d261b123a3a5223033cd1b11f42c1
I20260806 13:07:06.728255 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.723127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35293 (local address 127.13.224.66:56349)
0806 13:07:06.723272 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:06.723281 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:06.723331 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:06.723437 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:06.723438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:06.723636 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:06.723703 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.723705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.724486 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.724487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.724998 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.725000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.725302 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.725304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.725361 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.725678 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:06.725683 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:06.726003 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:06.727302 (+ 1299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:06.727303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:06.727303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:06.727503 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:06.727660 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:06.727661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:06.727662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:06.727727 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:06.727993 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:06.727993 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:06.728029 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 13:07:06.728093 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 13:07:06.728187 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":6,"thread_start_us":68,"threads_started":1}
I20260806 13:07:06.728266 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.723179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:56349 (local address 127.13.224.126:35293)
0806 13:07:06.723216 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.723218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.723340 (+ 122us) server_negotiation.cc:407] Connection header received
0806 13:07:06.723378 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.723379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.723389 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.723433 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:06.723780 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.724397 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.725069 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.725238 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.725285 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.726100 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:06.726101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:06.726101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:06.726109 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:06.727227 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.727567 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.727567 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.727568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.727609 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.727779 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.727779 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.727779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.727934 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:06.727996 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.728122 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.728189 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for oryx/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab.
Entry for principal oryx/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab.
I20260806 13:07:06.746495 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35293
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab
--principal=oryx/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:06.825891 25802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:06.826116 25802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:06.826145 25802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:06.827279 25802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:06.827320 25802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:06.827340 25802 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 13:07:06.827370 25802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:07:06.828879 25802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab
--principal=oryx/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.224.126:35293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:06.829084 25802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:06.829285 25802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:06.834079 25814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:06.834059 25810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:06.834151 25811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:06.834225 25802 server_base.cc:1034] running on GCE node
I20260806 13:07:06.834447 25802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:06.834666 25802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:06.835804 25802 hybrid_clock.cc:648] HybridClock initialized: now 1786021626835787 us; error 28 us; skew 500 ppm
Aug 06 13:07:06 dist-test-slave-q9h9 krb5kdc[25245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021626, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:06.838125 25802 init.cc:377] Logged in from keytab as oryx/127.13.224.67@KRBTEST.COM (short username oryx)
I20260806 13:07:06.838563 25802 webserver.cc:460] Webserver started at http://127.13.224.67:33241/ using document root <none> and password file <none>
I20260806 13:07:06.838762 25802 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:06.838809 25802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:06.838929 25802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:06.839882 25802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "10e53b6569344c35be2bd90a5975616a"
format_stamp: "Formatted at 2026-08-06 13:07:06 on dist-test-slave-q9h9"
server_key: "f88e175209a5462e31ac10706c91a37e"
server_key_iv: "809df47acca3adf1ad49e2ac4d3f5089"
server_key_version: "encryptionkey@0"
I20260806 13:07:06.840207 25802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "10e53b6569344c35be2bd90a5975616a"
format_stamp: "Formatted at 2026-08-06 13:07:06 on dist-test-slave-q9h9"
server_key: "f88e175209a5462e31ac10706c91a37e"
server_key_iv: "809df47acca3adf1ad49e2ac4d3f5089"
server_key_version: "encryptionkey@0"
I20260806 13:07:06.841423 25802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:07:06.842149 25825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:06.842325 25802 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:06.842409 25802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "10e53b6569344c35be2bd90a5975616a"
format_stamp: "Formatted at 2026-08-06 13:07:06 on dist-test-slave-q9h9"
server_key: "f88e175209a5462e31ac10706c91a37e"
server_key_iv: "809df47acca3adf1ad49e2ac4d3f5089"
server_key_version: "encryptionkey@0"
I20260806 13:07:06.842474 25802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:06.861631 25802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:06.863492 25802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:06.863624 25802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:06.864113 25802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:06.864152 25802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:06.864180 25802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:06.864205 25802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 13:07:06 dist-test-slave-q9h9 krb5kdc[25245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021626, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
I20260806 13:07:06.870877 25802 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:35123
I20260806 13:07:06.871181 26000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:35123 every 8 connection(s)
I20260806 13:07:06.871269 25802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:07:06.873145 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.864457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:53137 (local address 127.13.224.126:35293)
0806 13:07:06.864509 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.864512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.864691 (+ 179us) server_negotiation.cc:407] Connection header received
0806 13:07:06.865138 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.865140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.865153 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.865212 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:06.865790 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.866503 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.867445 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.867652 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.867717 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.870624 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:06.870625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:06.870626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:06.870634 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:06.872044 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.872430 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.872431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.872432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.872483 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.872661 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.872661 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.872662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.872760 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:06.872804 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.872994 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.873058 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 13:07:06.873298 25841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.864434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35293 (local address 127.13.224.67:53137)
0806 13:07:06.864570 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:06.864608 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:06.865028 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:06.865268 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:06.865271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:06.865492 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:06.865703 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.865708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.866587 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.866588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.867366 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.867368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.867722 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.867723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.867841 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.868268 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:06.868279 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:06.870531 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:06.872133 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:06.872138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:06.872140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:06.872355 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:06.872527 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:06.872528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:06.872528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:06.872625 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:06.872800 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:06.872803 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:06.872899 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 13:07:06.872967 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 13:07:06.873118 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":72,"threads_started":1}
I20260806 13:07:06.873740 26002 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35293
I20260806 13:07:06.873821 26002 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:06.873986 26002 heartbeater.cc:511] Master 127.13.224.126:35293 requested a full tablet report, sending...
I20260806 13:07:06.874373 25302 ts_manager.cc:194] Registered new tserver with Master: 10e53b6569344c35be2bd90a5975616a (127.13.224.67:35123)
I20260806 13:07:06.874855 25302 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:53137
I20260806 13:07:06.879581 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.874686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35293 (local address 127.13.224.67:34417)
0806 13:07:06.874818 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:06.874827 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:06.874879 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:06.875018 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:06.875019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:06.875171 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:06.875225 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.875226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.875863 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.875864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.876360 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.876362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.876684 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.876685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.876746 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.877098 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:06.877103 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:06.877351 (+ 248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:06.878706 (+ 1355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:06.878707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:06.878707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:06.878860 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:06.879021 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:06.879022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:06.879022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:06.879086 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:06.879284 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:06.879285 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:06.879329 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 13:07:06.879387 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 13:07:06.879481 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":72,"threads_started":1}
I20260806 13:07:06.879567 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.874751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:34417 (local address 127.13.224.126:35293)
0806 13:07:06.874787 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.874789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.874871 (+ 82us) server_negotiation.cc:407] Connection header received
0806 13:07:06.874915 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.874915 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.874924 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.874977 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:06.875293 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.875773 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.876428 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.876627 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.876678 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.877439 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:06.877440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:06.877440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:06.877447 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:06.878634 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.878917 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.878918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.878919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.878970 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.879135 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.879136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.879136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.879233 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:06.879286 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.879414 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.879479 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:07:06.880865 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 25802
I20260806 13:07:06.880966 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:07:06.881150 14209 external_mini_cluster.cc:1428] Setting key d2a43d78238f6c041b863a5a46bb8954
I20260806 13:07:06.881825 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 13:07:06 dist-test-slave-q9h9 krb5kdc[25245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021626, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 13:07:06 dist-test-slave-q9h9 krb5kdc[25245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:06 dist-test-slave-q9h9 krb5kdc[25245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 13:07:06.902534 25371 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:06.897331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46886 (local address 127.13.224.126:35293)
0806 13:07:06.897381 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.897382 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.897390 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:07:06.897453 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.897453 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.897463 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.897506 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:06.898057 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.898540 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.899247 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.899427 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.899481 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.902460 (+ 2979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46886: BlockingRecv error: recv got EOF from 127.0.0.1:46886 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 06 13:07:06 dist-test-slave-q9h9 krb5kdc[25245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021626, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
I20260806 13:07:06.908540 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.903144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46900 (local address 127.13.224.126:35293)
0806 13:07:06.903184 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.903186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.903243 (+ 57us) server_negotiation.cc:407] Connection header received
0806 13:07:06.903278 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.903279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.903286 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.903331 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:06.903721 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.904152 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.904765 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.904930 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.904976 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.906810 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:06.906811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:06.906811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:06.906817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:06.907811 (+ 994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.908069 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.908069 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.908070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.908112 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:06.908227 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:06.908227 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:06.908228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:06.908308 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:06.908371 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.908434 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.908486 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:07:06.909909 25302 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:07:06.911535 25302 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:07:06.917413 26036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.913771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:39475 (local address 127.0.0.1:40416)
0806 13:07:06.913925 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:06.913963 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:06.914162 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:06.914395 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:06.914398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:06.914400 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:06.914457 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.914459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.915446 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.915447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.916368 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.916371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.917092 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.917093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.917203 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.917218 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 13:07:06.917281 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 13:07:06.917339 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":2,"thread_start_us":55,"threads_started":1}
I20260806 13:07:06.917433 26044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.913965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40416 (local address 127.13.224.66:39475)
0806 13:07:06.914153 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.914158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.914169 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:07:06.914300 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.914302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.914318 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.914383 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:06.914522 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.915374 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.916442 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.917010 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.917077 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.917154 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.917303 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.917339 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260806 13:07:06.918133 26042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.913789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:38237 (local address 127.0.0.1:39800)
0806 13:07:06.914220 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:06.914229 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:06.914270 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:06.914763 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:06.914764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:06.914766 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:06.914820 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.914821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.916101 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.916101 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.916870 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.916872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.917855 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.917856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.917914 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.917918 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:07:06.917968 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 13:07:06.918005 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":184,"threads_started":1}
I20260806 13:07:06.918162 26039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.913963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39800 (local address 127.13.224.65:38237)
0806 13:07:06.914619 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.914623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.914631 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:07:06.914668 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.914670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.914684 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.914759 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:06.914889 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.915998 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.916948 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.917689 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.917755 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.917822 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.917997 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.918065 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":93,"threads_started":1}
I20260806 13:07:06.918300 25699 tablet_service.cc:1467] Processing CreateTablet for tablet ddc1dad02a8b4ee39b8e81116c09d593 (DEFAULT_TABLE table=test-table [id=6d3141566fb14e12b1e2fcebdccb15b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:06.918674 25699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddc1dad02a8b4ee39b8e81116c09d593. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:06.918802 25499 tablet_service.cc:1467] Processing CreateTablet for tablet ddc1dad02a8b4ee39b8e81116c09d593 (DEFAULT_TABLE table=test-table [id=6d3141566fb14e12b1e2fcebdccb15b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:06.919154 26037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.913775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:35123 (local address 127.0.0.1:38490)
0806 13:07:06.914159 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:06.914169 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:06.914224 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:06.914847 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:06.914848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:06.914851 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:06.914892 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.914894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.916950 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.916951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.917987 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.917989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.918932 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.918933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.918992 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.918996 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:07:06.919053 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 13:07:06.919091 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":5,"thread_start_us":280,"threads_started":1}
I20260806 13:07:06.919153 25499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddc1dad02a8b4ee39b8e81116c09d593. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:06.919229 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.914155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38490 (local address 127.13.224.67:35123)
0806 13:07:06.914347 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.914351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.914376 (+ 25us) server_negotiation.cc:407] Connection header received
0806 13:07:06.914415 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.914417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.914433 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.914504 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:06.914969 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.916016 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.918107 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.918755 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.918822 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.918895 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.919087 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.919128 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":78,"threads_started":1}
I20260806 13:07:06.919793 25906 tablet_service.cc:1467] Processing CreateTablet for tablet ddc1dad02a8b4ee39b8e81116c09d593 (DEFAULT_TABLE table=test-table [id=6d3141566fb14e12b1e2fcebdccb15b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:06.920125 25906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddc1dad02a8b4ee39b8e81116c09d593. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:06.921890 26048 tablet_bootstrap.cc:492] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6: Bootstrap starting.
I20260806 13:07:06.922457 26049 tablet_bootstrap.cc:492] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b: Bootstrap starting.
I20260806 13:07:06.922570 26048 tablet_bootstrap.cc:654] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:06.922891 26048 log.cc:824] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:06.923189 26049 tablet_bootstrap.cc:654] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:06.923571 26049 log.cc:824] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:06.923729 26048 tablet_bootstrap.cc:492] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6: No bootstrap required, opened a new log
I20260806 13:07:06.923803 26048 ts_tablet_manager.cc:1397] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 13:07:06.923869 26051 tablet_bootstrap.cc:492] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a: Bootstrap starting.
I20260806 13:07:06.924350 26049 tablet_bootstrap.cc:492] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b: No bootstrap required, opened a new log
I20260806 13:07:06.924396 26051 tablet_bootstrap.cc:654] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:06.924409 26049 ts_tablet_manager.cc:1397] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:07:06.924700 26051 log.cc:824] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:06.925331 26048 raft_consensus.cc:348] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } }
I20260806 13:07:06.925458 26048 raft_consensus.cc:374] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:06.925462 26051 tablet_bootstrap.cc:492] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a: No bootstrap required, opened a new log
I20260806 13:07:06.925489 26048 raft_consensus.cc:729] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591f2c06adac47ae96d94bf1ed013bf6, State: Initialized, Role: FOLLOWER
I20260806 13:07:06.925518 26051 ts_tablet_manager.cc:1397] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:07:06.925602 26048 consensus_queue.cc:260] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } }
I20260806 13:07:06.925873 26048 ts_tablet_manager.cc:1428] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 13:07:06.925897 25592 heartbeater.cc:503] Master 127.13.224.126:35293 was elected leader, sending a full tablet report...
I20260806 13:07:06.926811 26049 raft_consensus.cc:348] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } }
I20260806 13:07:06.926966 26049 raft_consensus.cc:374] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:06.927059 26049 raft_consensus.cc:729] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba5659cf28047c090f6073d5779851b, State: Initialized, Role: FOLLOWER
I20260806 13:07:06.927170 26049 consensus_queue.cc:260] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } }
I20260806 13:07:06.927438 26049 ts_tablet_manager.cc:1428] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:07:06.927464 25790 heartbeater.cc:503] Master 127.13.224.126:35293 was elected leader, sending a full tablet report...
I20260806 13:07:06.927413 26051 raft_consensus.cc:348] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } }
I20260806 13:07:06.927557 26051 raft_consensus.cc:374] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:06.927597 26051 raft_consensus.cc:729] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10e53b6569344c35be2bd90a5975616a, State: Initialized, Role: FOLLOWER
I20260806 13:07:06.927698 26051 consensus_queue.cc:260] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } }
I20260806 13:07:06.927953 26051 ts_tablet_manager.cc:1428] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 13:07:06.928023 26002 heartbeater.cc:503] Master 127.13.224.126:35293 was elected leader, sending a full tablet report...
I20260806 13:07:06.936925 26058 raft_consensus.cc:482] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:07:06.936995 26058 raft_consensus.cc:504] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } }
I20260806 13:07:06.937269 26058 leader_election.cc:288] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dba5659cf28047c090f6073d5779851b (127.13.224.66:39475), 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237)
I20260806 13:07:06.939890 26039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.937485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:55007 (local address 127.13.224.65:38237)
0806 13:07:06.937519 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.937521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.937641 (+ 120us) server_negotiation.cc:407] Connection header received
0806 13:07:06.937685 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.937686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.937694 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.937743 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:06.937876 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.938343 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.939227 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.939600 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.939664 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.939672 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.939794 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.939832 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:07:06.939884 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.937401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:38237 (local address 127.13.224.67:55007)
0806 13:07:06.937576 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:06.937585 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:06.937640 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:06.937748 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:06.937749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:06.937751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:06.937820 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.937821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.938418 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.938419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.939148 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.939150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.939674 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.939675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.939736 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.939739 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:07:06.939780 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 13:07:06.939809 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":44,"thread_start_us":81,"threads_started":1}
I20260806 13:07:06.939925 26044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.937465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:41389 (local address 127.13.224.66:39475)
0806 13:07:06.937501 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.937503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.937512 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:06.937535 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.937536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.937544 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.937608 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:06.937752 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.938290 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.939252 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.939592 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.939649 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.939656 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.939835 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.939872 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:07:06.939966 25841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.937401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:39475 (local address 127.13.224.67:41389)
0806 13:07:06.937439 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:06.937450 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:06.937487 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:06.937615 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:06.937616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:06.937617 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:06.937677 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.937679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.938355 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.938356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.939143 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:06.939145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.939661 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.939662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:06.939767 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.939770 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:07:06.939822 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 13:07:06.939857 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":38}
I20260806 13:07:06.940234 25528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddc1dad02a8b4ee39b8e81116c09d593" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591f2c06adac47ae96d94bf1ed013bf6" is_pre_election: true
I20260806 13:07:06.940248 25727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddc1dad02a8b4ee39b8e81116c09d593" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dba5659cf28047c090f6073d5779851b" is_pre_election: true
I20260806 13:07:06.940359 25528 raft_consensus.cc:2454] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 0.
I20260806 13:07:06.940363 25727 raft_consensus.cc:2454] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 0.
I20260806 13:07:06.940578 25826 leader_election.cc:302] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10e53b6569344c35be2bd90a5975616a, 591f2c06adac47ae96d94bf1ed013bf6; no voters:
I20260806 13:07:06.940685 26058 raft_consensus.cc:2781] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:07:06.940747 26058 raft_consensus.cc:482] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:07:06.940763 26058 raft_consensus.cc:3037] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:06.941543 26058 raft_consensus.cc:504] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } }
I20260806 13:07:06.941658 26058 leader_election.cc:288] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 election: Requested vote from peers dba5659cf28047c090f6073d5779851b (127.13.224.66:39475), 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237)
I20260806 13:07:06.941795 25727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddc1dad02a8b4ee39b8e81116c09d593" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dba5659cf28047c090f6073d5779851b"
I20260806 13:07:06.941797 25528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddc1dad02a8b4ee39b8e81116c09d593" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591f2c06adac47ae96d94bf1ed013bf6"
I20260806 13:07:06.941866 25528 raft_consensus.cc:3037] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:06.941866 25727 raft_consensus.cc:3037] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:06.942660 25528 raft_consensus.cc:2454] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 1.
I20260806 13:07:06.942660 25727 raft_consensus.cc:2454] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 1.
I20260806 13:07:06.942826 25828 leader_election.cc:302] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10e53b6569344c35be2bd90a5975616a, dba5659cf28047c090f6073d5779851b; no voters:
I20260806 13:07:06.942930 26058 raft_consensus.cc:2781] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:07:06.943058 26058 raft_consensus.cc:686] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [term 1 LEADER]: Becoming Leader. State: Replica: 10e53b6569344c35be2bd90a5975616a, State: Running, Role: LEADER
I20260806 13:07:06.943128 26058 consensus_queue.cc:237] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } }
I20260806 13:07:06.943881 25302 catalog_manager.cc:5495] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a reported cstate change: term changed from 0 to 1, leader changed from <none> to 10e53b6569344c35be2bd90a5975616a (127.13.224.67). New cstate: current_term: 1 leader_uuid: "10e53b6569344c35be2bd90a5975616a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } health_report { overall_health: UNKNOWN } } }
I20260806 13:07:06.954062 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:06.951935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38496 (local address 127.13.224.67:35123)
0806 13:07:06.951989 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:06.951990 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:06.951999 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:06.952028 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:06.952029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:06.952045 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:06.952105 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:07:06.952231 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.952661 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.953360 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:06.953551 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:06.953603 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:06.953769 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:07:06.953853 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:07:06.953916 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:06.953956 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 13:07:06.953975 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 13:07:06.955577 25727 raft_consensus.cc:1261] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Refusing update from remote peer 10e53b6569344c35be2bd90a5975616a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:06.955601 25528 raft_consensus.cc:1261] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Refusing update from remote peer 10e53b6569344c35be2bd90a5975616a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:06.955890 26061 consensus_queue.cc:1035] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [LEADER]: Connected to new peer: Peer: permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:06.956003 26058 consensus_queue.cc:1035] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:06.958528 26066 mvcc.cc:205] Tried to move back new op lower bound from 7315544584007860224 to 7315544583959662592. Current Snapshot: MvccSnapshot[applied={T|T < 7315544584007860224}]
I20260806 13:07:06.959010 26067 mvcc.cc:205] Tried to move back new op lower bound from 7315544584007860224 to 7315544583959662592. Current Snapshot: MvccSnapshot[applied={T|T < 7315544584007860224}]
I20260806 13:07:06.960230 25301 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:06.960307 25301 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:06.960911 25301 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8: Sending DeleteTablet for 3 replicas of tablet ddc1dad02a8b4ee39b8e81116c09d593
I20260806 13:07:06.961268 25906 tablet_service.cc:1514] Processing DeleteTablet for tablet ddc1dad02a8b4ee39b8e81116c09d593 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:06 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:38490
I20260806 13:07:06.961251 25499 tablet_service.cc:1514] Processing DeleteTablet for tablet ddc1dad02a8b4ee39b8e81116c09d593 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:06 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:39800
I20260806 13:07:06.961468 25699 tablet_service.cc:1514] Processing DeleteTablet for tablet ddc1dad02a8b4ee39b8e81116c09d593 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:06 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:40416
I20260806 13:07:06.961555 25301 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 13:07:06.961676 26086 tablet_replica.cc:330] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b: stopping tablet replica
W20260806 13:07:06.961697 25301 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:07:06.961758 26086 raft_consensus.cc:2229] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:06.961623 26084 tablet_replica.cc:330] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a: stopping tablet replica
I20260806 13:07:06.961946 26084 raft_consensus.cc:2229] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:07:06.962188 26085 tablet_replica.cc:330] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6: stopping tablet replica
I20260806 13:07:06.962241 26086 raft_consensus.cc:2258] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:06.962371 26084 raft_consensus.cc:2258] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:06.962550 26085 raft_consensus.cc:2229] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:06.962726 26085 raft_consensus.cc:2258] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:06.963006 25699 tablet_service.cc:1467] Processing CreateTablet for tablet 21ffc3de044b42f2932ecee8ff4cbc24 (DEFAULT_TABLE table=test-table [id=dbfd7201c2924caeaeb2c29bb0331e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:06.963135 25699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ffc3de044b42f2932ecee8ff4cbc24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:06.963116 25499 tablet_service.cc:1467] Processing CreateTablet for tablet 21ffc3de044b42f2932ecee8ff4cbc24 (DEFAULT_TABLE table=test-table [id=dbfd7201c2924caeaeb2c29bb0331e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:06.963215 25499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ffc3de044b42f2932ecee8ff4cbc24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:06.963433 26085 ts_tablet_manager.cc:1905] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:06.964383 26049 tablet_bootstrap.cc:492] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b: Bootstrap starting.
I20260806 13:07:06.964498 26048 tablet_bootstrap.cc:492] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6: Bootstrap starting.
I20260806 13:07:06.964871 26049 tablet_bootstrap.cc:654] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:06.964871 26048 tablet_bootstrap.cc:654] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:06.965265 26085 ts_tablet_manager.cc:1918] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:07:06.965291 25906 tablet_service.cc:1467] Processing CreateTablet for tablet 21ffc3de044b42f2932ecee8ff4cbc24 (DEFAULT_TABLE table=test-table [id=dbfd7201c2924caeaeb2c29bb0331e71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:06.965389 25906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ffc3de044b42f2932ecee8ff4cbc24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:06.965462 26085 log.cc:1196] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/wal/wals/ddc1dad02a8b4ee39b8e81116c09d593
I20260806 13:07:06.965770 26085 ts_tablet_manager.cc:1939] T ddc1dad02a8b4ee39b8e81116c09d593 P 591f2c06adac47ae96d94bf1ed013bf6: Deleting consensus metadata
I20260806 13:07:06.966236 25282 catalog_manager.cc:4841] TS 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237): tablet ddc1dad02a8b4ee39b8e81116c09d593 (table test-table [id=6d3141566fb14e12b1e2fcebdccb15b6]) successfully deleted
I20260806 13:07:06.966495 26051 tablet_bootstrap.cc:492] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a: Bootstrap starting.
I20260806 13:07:06.966912 26084 ts_tablet_manager.cc:1905] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:06.966925 26051 tablet_bootstrap.cc:654] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:06.967629 26051 tablet_bootstrap.cc:492] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a: No bootstrap required, opened a new log
I20260806 13:07:06.967684 26051 ts_tablet_manager.cc:1397] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:06.967818 26051 raft_consensus.cc:348] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:06.967885 26051 raft_consensus.cc:374] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:06.967903 26051 raft_consensus.cc:729] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10e53b6569344c35be2bd90a5975616a, State: Initialized, Role: FOLLOWER
I20260806 13:07:06.967952 26051 consensus_queue.cc:260] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:06.968060 26051 ts_tablet_manager.cc:1428] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:06.968642 26086 ts_tablet_manager.cc:1905] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:06.969731 26086 ts_tablet_manager.cc:1918] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:07:06.969789 26086 log.cc:1196] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/ddc1dad02a8b4ee39b8e81116c09d593
I20260806 13:07:06.969946 26086 ts_tablet_manager.cc:1939] T ddc1dad02a8b4ee39b8e81116c09d593 P dba5659cf28047c090f6073d5779851b: Deleting consensus metadata
I20260806 13:07:06.970289 25283 catalog_manager.cc:4841] TS dba5659cf28047c090f6073d5779851b (127.13.224.66:39475): tablet ddc1dad02a8b4ee39b8e81116c09d593 (table test-table [id=6d3141566fb14e12b1e2fcebdccb15b6]) successfully deleted
I20260806 13:07:06.970547 26084 ts_tablet_manager.cc:1918] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:07:06.970606 26084 log.cc:1196] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/ddc1dad02a8b4ee39b8e81116c09d593
I20260806 13:07:06.970774 26084 ts_tablet_manager.cc:1939] T ddc1dad02a8b4ee39b8e81116c09d593 P 10e53b6569344c35be2bd90a5975616a: Deleting consensus metadata
I20260806 13:07:06.971156 26049 tablet_bootstrap.cc:492] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b: No bootstrap required, opened a new log
I20260806 13:07:06.971205 26049 ts_tablet_manager.cc:1397] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260806 13:07:06.971194 25284 catalog_manager.cc:4841] TS 10e53b6569344c35be2bd90a5975616a (127.13.224.67:35123): tablet ddc1dad02a8b4ee39b8e81116c09d593 (table test-table [id=6d3141566fb14e12b1e2fcebdccb15b6]) successfully deleted
I20260806 13:07:06.971340 26049 raft_consensus.cc:348] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:06.971393 26049 raft_consensus.cc:374] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:06.971405 26049 raft_consensus.cc:729] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba5659cf28047c090f6073d5779851b, State: Initialized, Role: FOLLOWER
I20260806 13:07:06.971437 26049 consensus_queue.cc:260] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:06.971518 26049 ts_tablet_manager.cc:1428] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:06.971930 26048 tablet_bootstrap.cc:492] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6: No bootstrap required, opened a new log
I20260806 13:07:06.971978 26048 ts_tablet_manager.cc:1397] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260806 13:07:06.972102 26048 raft_consensus.cc:348] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:06.972162 26048 raft_consensus.cc:374] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:06.972173 26048 raft_consensus.cc:729] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591f2c06adac47ae96d94bf1ed013bf6, State: Initialized, Role: FOLLOWER
I20260806 13:07:06.972204 26048 consensus_queue.cc:260] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:06.972281 26048 ts_tablet_manager.cc:1428] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:07.063462 26056 raft_consensus.cc:482] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:07:07.063560 26056 raft_consensus.cc:504] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.063894 26056 leader_election.cc:288] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237), 10e53b6569344c35be2bd90a5975616a (127.13.224.67:35123)
I20260806 13:07:07.067175 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.064157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:55537 (local address 127.13.224.67:35123)
0806 13:07:07.064204 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:07.064206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:07.064216 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:07:07.064244 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:07.064245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:07.064255 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:07.064306 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:07.064495 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.065003 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.066279 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.066849 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.066912 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.066921 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:07.067070 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 13:07:07.067102 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:07:07.067185 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.064066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:35123 (local address 127.13.224.66:55537)
0806 13:07:07.064131 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:07.064142 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:07.064185 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:07.064358 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:07.064359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:07.064362 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:07.064434 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.064436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.065074 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.065076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.066187 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.066190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.066925 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.066926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.067006 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.067009 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:07:07.067060 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 13:07:07.067104 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 13:07:07.067241 26039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.064286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:38837 (local address 127.13.224.65:38237)
0806 13:07:07.064331 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:07.064333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:07.064340 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:07.064366 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:07.064367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:07.064375 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:07.064415 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:07.064625 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.065361 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.066531 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.066890 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.066959 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.066968 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:07.067135 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 13:07:07.067181 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:07:07.067427 25936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ffc3de044b42f2932ecee8ff4cbc24" candidate_uuid: "dba5659cf28047c090f6073d5779851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10e53b6569344c35be2bd90a5975616a" is_pre_election: true
I20260806 13:07:07.067544 25936 raft_consensus.cc:2454] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dba5659cf28047c090f6073d5779851b in term 0.
I20260806 13:07:07.067610 26091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.064067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:38237 (local address 127.13.224.66:38837)
0806 13:07:07.064240 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:07.064248 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:07.064291 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:07.064438 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:07.064438 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:07.064440 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:07.064543 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.064545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.065450 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.065451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.066457 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.066459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.066997 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.066998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.067058 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.067061 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:07:07.067466 (+ 405us) client_negotiation.cc:240] Negotiation successful
0806 13:07:07.067503 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":43,"thread_start_us":66,"threads_started":1}
I20260806 13:07:07.067724 25633 leader_election.cc:302] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10e53b6569344c35be2bd90a5975616a, dba5659cf28047c090f6073d5779851b; no voters:
I20260806 13:07:07.067844 26056 raft_consensus.cc:2781] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:07:07.067881 26056 raft_consensus.cc:482] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:07:07.067899 26056 raft_consensus.cc:3037] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:07.067915 25528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ffc3de044b42f2932ecee8ff4cbc24" candidate_uuid: "dba5659cf28047c090f6073d5779851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591f2c06adac47ae96d94bf1ed013bf6" is_pre_election: true
I20260806 13:07:07.067996 25528 raft_consensus.cc:2454] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dba5659cf28047c090f6073d5779851b in term 0.
I20260806 13:07:07.068635 26056 raft_consensus.cc:504] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.068760 26056 leader_election.cc:288] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [CANDIDATE]: Term 1 election: Requested vote from peers 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237), 10e53b6569344c35be2bd90a5975616a (127.13.224.67:35123)
I20260806 13:07:07.068945 25528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ffc3de044b42f2932ecee8ff4cbc24" candidate_uuid: "dba5659cf28047c090f6073d5779851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591f2c06adac47ae96d94bf1ed013bf6"
I20260806 13:07:07.068954 25936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ffc3de044b42f2932ecee8ff4cbc24" candidate_uuid: "dba5659cf28047c090f6073d5779851b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10e53b6569344c35be2bd90a5975616a"
I20260806 13:07:07.069013 25528 raft_consensus.cc:3037] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:07.069039 25936 raft_consensus.cc:3037] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:07.069584 25528 raft_consensus.cc:2454] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dba5659cf28047c090f6073d5779851b in term 1.
I20260806 13:07:07.069751 25631 leader_election.cc:302] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 591f2c06adac47ae96d94bf1ed013bf6, dba5659cf28047c090f6073d5779851b; no voters:
I20260806 13:07:07.069783 25936 raft_consensus.cc:2454] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dba5659cf28047c090f6073d5779851b in term 1.
I20260806 13:07:07.069828 26056 raft_consensus.cc:2781] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:07:07.069948 26056 raft_consensus.cc:686] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [term 1 LEADER]: Becoming Leader. State: Replica: dba5659cf28047c090f6073d5779851b, State: Running, Role: LEADER
I20260806 13:07:07.070041 26056 consensus_queue.cc:237] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.070748 25301 catalog_manager.cc:5495] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b reported cstate change: term changed from 0 to 1, leader changed from <none> to dba5659cf28047c090f6073d5779851b (127.13.224.66). New cstate: current_term: 1 leader_uuid: "dba5659cf28047c090f6073d5779851b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } health_report { overall_health: UNKNOWN } } }
W20260806 13:07:07.071311 25594 tablet.cc:2369] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 13:07:07 dist-test-slave-q9h9 krb5kdc[25245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021626, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.126@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
I20260806 13:07:07.108955 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.102746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46910 (local address 127.13.224.126:35293)
0806 13:07:07.102805 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:07.102807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:07.102918 (+ 111us) server_negotiation.cc:407] Connection header received
0806 13:07:07.102964 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:07.102965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:07.102974 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:07.103019 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:07.103482 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.103958 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.104590 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.104759 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.104802 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.106796 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:07.106798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:07.106799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:07.106805 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:07.107883 (+ 1078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:07.108214 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:07.108214 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:07.108215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:07.108276 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:07.108431 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:07.108431 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:07.108432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:07.108530 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:07.108584 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:07.108721 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 13:07:07.108811 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":32}
I20260806 13:07:07.109022 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.102698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35293 (local address 127.0.0.1:46910)
0806 13:07:07.102850 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:07.102859 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:07.102923 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:07.103037 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:07.103038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:07.103371 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:07.103419 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.103421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.104019 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.104019 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.104538 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.104539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.104804 (+ 265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.104805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.104840 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.105332 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:07.105336 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:07.106709 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:07.107955 (+ 1246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:07.107964 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:07.107967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:07.108150 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:07.108329 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:07.108329 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:07.108330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:07.108381 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:07.108581 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:07.108590 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:07.108636 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 13:07:07.108694 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 13:07:07.108876 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":70,"thread_start_us":100,"threads_started":1}
I20260806 13:07:07.110939 25301 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:46910:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 13:07:07.111130 25301 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:07:07.112468 25906 tablet_service.cc:1467] Processing CreateTablet for tablet aed9f91896664ecfa005f2a9eedbc9dd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b84073878cf24235a271edfd2963f45a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 13:07:07.112457 25499 tablet_service.cc:1467] Processing CreateTablet for tablet aed9f91896664ecfa005f2a9eedbc9dd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b84073878cf24235a271edfd2963f45a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 13:07:07.112457 25699 tablet_service.cc:1467] Processing CreateTablet for tablet aed9f91896664ecfa005f2a9eedbc9dd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b84073878cf24235a271edfd2963f45a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 13:07:07.112624 25499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aed9f91896664ecfa005f2a9eedbc9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:07.112624 25699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aed9f91896664ecfa005f2a9eedbc9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:07.112627 25906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aed9f91896664ecfa005f2a9eedbc9dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:07.113922 26051 tablet_bootstrap.cc:492] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a: Bootstrap starting.
I20260806 13:07:07.114113 26049 tablet_bootstrap.cc:492] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b: Bootstrap starting.
I20260806 13:07:07.114112 26048 tablet_bootstrap.cc:492] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6: Bootstrap starting.
I20260806 13:07:07.114307 26051 tablet_bootstrap.cc:654] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:07.114517 26048 tablet_bootstrap.cc:654] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:07.114517 26049 tablet_bootstrap.cc:654] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:07.114825 26051 tablet_bootstrap.cc:492] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a: No bootstrap required, opened a new log
I20260806 13:07:07.114881 26051 ts_tablet_manager.cc:1397] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:07.114995 26051 raft_consensus.cc:348] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.115070 26051 raft_consensus.cc:374] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:07.115091 26051 raft_consensus.cc:729] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10e53b6569344c35be2bd90a5975616a, State: Initialized, Role: FOLLOWER
I20260806 13:07:07.115134 26051 consensus_queue.cc:260] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.115208 26048 tablet_bootstrap.cc:492] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6: No bootstrap required, opened a new log
I20260806 13:07:07.115231 26051 ts_tablet_manager.cc:1428] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:07.115263 26048 ts_tablet_manager.cc:1397] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 13:07:07.115243 26061 tablet_replica.cc:439] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } }
I20260806 13:07:07.115434 26061 tablet_replica.cc:442] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:07:07.115406 26048 raft_consensus.cc:348] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.115473 26048 raft_consensus.cc:374] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:07.115492 26048 raft_consensus.cc:729] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591f2c06adac47ae96d94bf1ed013bf6, State: Initialized, Role: FOLLOWER
I20260806 13:07:07.115530 26048 consensus_queue.cc:260] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.115605 26048 ts_tablet_manager.cc:1428] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:07.115731 26054 tablet_replica.cc:439] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } }
I20260806 13:07:07.115800 26054 tablet_replica.cc:442] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:07:07.116003 26049 tablet_bootstrap.cc:492] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b: No bootstrap required, opened a new log
I20260806 13:07:07.116048 26049 ts_tablet_manager.cc:1397] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:07:07.116374 26049 raft_consensus.cc:348] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.116449 26049 raft_consensus.cc:374] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:07.116468 26049 raft_consensus.cc:729] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba5659cf28047c090f6073d5779851b, State: Initialized, Role: FOLLOWER
I20260806 13:07:07.116524 26049 consensus_queue.cc:260] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.116616 26049 ts_tablet_manager.cc:1428] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 13:07:07.116657 26056 tablet_replica.cc:439] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } }
I20260806 13:07:07.116767 26056 tablet_replica.cc:442] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 13:07:07.122330 26003 tablet.cc:2369] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:07:07.190497 26061 raft_consensus.cc:482] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:07:07.190600 26061 raft_consensus.cc:504] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.190775 26061 leader_election.cc:288] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237), dba5659cf28047c090f6073d5779851b (127.13.224.66:39475)
I20260806 13:07:07.191008 25727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aed9f91896664ecfa005f2a9eedbc9dd" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dba5659cf28047c090f6073d5779851b" is_pre_election: true
I20260806 13:07:07.191008 25528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aed9f91896664ecfa005f2a9eedbc9dd" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591f2c06adac47ae96d94bf1ed013bf6" is_pre_election: true
I20260806 13:07:07.191123 25727 raft_consensus.cc:2454] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 0.
I20260806 13:07:07.191123 25528 raft_consensus.cc:2454] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 0.
I20260806 13:07:07.191267 25826 leader_election.cc:302] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10e53b6569344c35be2bd90a5975616a, 591f2c06adac47ae96d94bf1ed013bf6; no voters:
I20260806 13:07:07.191360 26061 raft_consensus.cc:2781] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:07:07.191416 26061 raft_consensus.cc:482] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:07:07.191437 26061 raft_consensus.cc:3037] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:07.192169 26061 raft_consensus.cc:504] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.192298 26061 leader_election.cc:288] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 election: Requested vote from peers 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237), dba5659cf28047c090f6073d5779851b (127.13.224.66:39475)
I20260806 13:07:07.192454 25727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aed9f91896664ecfa005f2a9eedbc9dd" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dba5659cf28047c090f6073d5779851b"
I20260806 13:07:07.192464 25528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aed9f91896664ecfa005f2a9eedbc9dd" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591f2c06adac47ae96d94bf1ed013bf6"
I20260806 13:07:07.192528 25727 raft_consensus.cc:3037] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:07.192541 25528 raft_consensus.cc:3037] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:07.193091 25528 raft_consensus.cc:2454] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 1.
I20260806 13:07:07.193221 25727 raft_consensus.cc:2454] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 1.
I20260806 13:07:07.193286 25826 leader_election.cc:302] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10e53b6569344c35be2bd90a5975616a, 591f2c06adac47ae96d94bf1ed013bf6; no voters:
I20260806 13:07:07.193365 26061 raft_consensus.cc:2781] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:07:07.193418 26061 raft_consensus.cc:686] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [term 1 LEADER]: Becoming Leader. State: Replica: 10e53b6569344c35be2bd90a5975616a, State: Running, Role: LEADER
I20260806 13:07:07.193471 26061 consensus_queue.cc:237] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.193760 26077 tablet_replica.cc:439] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a: TxnStatusTablet state changed. Reason: New leader 10e53b6569344c35be2bd90a5975616a. Latest consensus state: current_term: 1 leader_uuid: "10e53b6569344c35be2bd90a5975616a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } }
I20260806 13:07:07.193841 26077 tablet_replica.cc:442] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a: This TxnStatusTablet replica's current role is: LEADER
I20260806 13:07:07.194080 26110 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 13:07:07.194080 25301 catalog_manager.cc:5495] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a reported cstate change: term changed from 0 to 1, leader changed from <none> to 10e53b6569344c35be2bd90a5975616a (127.13.224.67). New cstate: current_term: 1 leader_uuid: "10e53b6569344c35be2bd90a5975616a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } health_report { overall_health: HEALTHY } } }
I20260806 13:07:07.194195 26110 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 13:07:07.207298 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.205067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:35123 (local address 127.0.0.1:38500)
0806 13:07:07.205132 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:07.205144 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:07.205193 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:07.205330 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:07.205331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:07.205332 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 13:07:07.205380 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.205382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.205969 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.205970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.206570 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.206572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.206902 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.206903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.206969 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.206985 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 13:07:07.207136 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 13:07:07.207144 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 13:07:07.207180 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 13:07:07.207229 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 13:07:07.207296 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.205134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38500 (local address 127.13.224.67:35123)
0806 13:07:07.205182 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:07.205184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:07.205193 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:07.205261 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:07.205262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:07.205273 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:07.205314 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:07:07.205433 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.205900 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.206644 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.206843 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.206898 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.207052 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:07:07.207088 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:07:07.207129 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:07.207193 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 13:07:07.207214 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:07:07.208261 25727 raft_consensus.cc:1261] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Refusing update from remote peer 10e53b6569344c35be2bd90a5975616a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:07.208292 25528 raft_consensus.cc:1261] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Refusing update from remote peer 10e53b6569344c35be2bd90a5975616a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:07.208457 26077 consensus_queue.cc:1035] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:07.208554 26061 consensus_queue.cc:1035] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a [LEADER]: Connected to new peer: Peer: permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:07.209174 26056 tablet_replica.cc:439] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b: TxnStatusTablet state changed. Reason: New leader 10e53b6569344c35be2bd90a5975616a. Latest consensus state: current_term: 1 leader_uuid: "10e53b6569344c35be2bd90a5975616a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } }
I20260806 13:07:07.209272 26056 tablet_replica.cc:442] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:07:07.209573 26058 tablet_replica.cc:439] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "10e53b6569344c35be2bd90a5975616a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } }
I20260806 13:07:07.209646 26058 tablet_replica.cc:442] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a: This TxnStatusTablet replica's current role is: LEADER
I20260806 13:07:07.209694 26065 mvcc.cc:205] Tried to move back new op lower bound from 7315544585043451904 to 7315544584984825856. Current Snapshot: MvccSnapshot[applied={T|T < 7315544585043451904}]
I20260806 13:07:07.210124 26054 tablet_replica.cc:439] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6: TxnStatusTablet state changed. Reason: New leader 10e53b6569344c35be2bd90a5975616a. Latest consensus state: current_term: 1 leader_uuid: "10e53b6569344c35be2bd90a5975616a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } }
I20260806 13:07:07.210211 26054 tablet_replica.cc:442] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:07:07.210254 26054 tablet_replica.cc:439] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "10e53b6569344c35be2bd90a5975616a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } }
I20260806 13:07:07.210292 26054 tablet_replica.cc:442] T aed9f91896664ecfa005f2a9eedbc9dd P 591f2c06adac47ae96d94bf1ed013bf6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:07:07.210608 26061 tablet_replica.cc:439] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "10e53b6569344c35be2bd90a5975616a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } }
I20260806 13:07:07.210683 26061 tablet_replica.cc:442] T aed9f91896664ecfa005f2a9eedbc9dd P 10e53b6569344c35be2bd90a5975616a: This TxnStatusTablet replica's current role is: LEADER
I20260806 13:07:07.210925 26056 tablet_replica.cc:439] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "10e53b6569344c35be2bd90a5975616a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } } }
I20260806 13:07:07.211004 26056 tablet_replica.cc:442] T aed9f91896664ecfa005f2a9eedbc9dd P dba5659cf28047c090f6073d5779851b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 13:07:07.215026 26044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.212627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40418 (local address 127.13.224.66:39475)
0806 13:07:07.212662 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:07.212664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:07.212670 (+ 6us) server_negotiation.cc:407] Connection header received
0806 13:07:07.212701 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:07.212702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:07.212718 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:07.212778 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:07:07.212905 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.213345 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.214395 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.214568 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.214612 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.214750 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:07:07.214851 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:07:07.214898 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:07.214949 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 13:07:07.214963 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:07:07.219575 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.217340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:44025 (local address 127.13.224.67:35123)
0806 13:07:07.217376 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:07.217378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:07.217385 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:07.217447 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:07.217448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:07.217457 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:07.217499 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:07:07.217606 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.218054 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.219046 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.219213 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.219254 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.219417 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:07:07.219439 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:07:07.219468 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:07.219516 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 13:07:07.219530 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:07:07.219623 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.217282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:35123 (local address 127.13.224.66:44025)
0806 13:07:07.217350 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:07.217358 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:07.217393 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:07.217504 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:07.217504 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:07.217505 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 13:07:07.217557 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.217558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.218103 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.218104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.218977 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.218980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.219270 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.219271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.219337 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.219352 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 13:07:07.219471 (+ 119us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 13:07:07.219472 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 13:07:07.219514 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 13:07:07.219552 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 13:07:07.219810 25791 tablet.cc:2369] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:07:07.222101 25528 raft_consensus.cc:1261] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Refusing update from remote peer dba5659cf28047c090f6073d5779851b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:07.222158 25936 raft_consensus.cc:1261] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Refusing update from remote peer dba5659cf28047c090f6073d5779851b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:07.222448 26056 consensus_queue.cc:1035] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [LEADER]: Connected to new peer: Peer: permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:07.222610 26094 consensus_queue.cc:1035] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [LEADER]: Connected to new peer: Peer: permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:07.230273 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.227671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:39475 (local address 127.13.224.67:37249)
0806 13:07:07.227773 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:07.227782 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:07.227817 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:07.228001 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:07.228002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:07.228003 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 13:07:07.228042 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.228043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.228576 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.228576 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.229451 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.229453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.229913 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.229914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.229961 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.229975 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 13:07:07.230160 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 13:07:07.230162 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:07:07.230191 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 13:07:07.230223 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 13:07:07.230286 26044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.227882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:37249 (local address 127.13.224.66:39475)
0806 13:07:07.227920 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:07.227922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:07.227929 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:07.227945 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:07.227946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:07.227955 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:07.227994 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:07:07.228097 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.228514 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.229546 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.229838 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.229902 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.230077 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:07:07.230108 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:07:07.230157 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:07.230194 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 13:07:07.230212 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:07:07.242882 25301 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:07.242975 25301 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:07.243568 25301 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8: Sending DeleteTablet for 3 replicas of tablet 21ffc3de044b42f2932ecee8ff4cbc24
I20260806 13:07:07.243847 25699 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ffc3de044b42f2932ecee8ff4cbc24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:07 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:40416
I20260806 13:07:07.243963 26086 tablet_replica.cc:330] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b: stopping tablet replica
I20260806 13:07:07.244009 26086 raft_consensus.cc:2229] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:07:07.244086 26086 raft_consensus.cc:2258] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:07.244264 25499 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ffc3de044b42f2932ecee8ff4cbc24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:07 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:39800
I20260806 13:07:07.244379 26085 tablet_replica.cc:330] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6: stopping tablet replica
I20260806 13:07:07.244446 26085 raft_consensus.cc:2229] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:07.244479 26085 raft_consensus.cc:2258] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:07.244815 26085 ts_tablet_manager.cc:1905] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:07.245185 26086 ts_tablet_manager.cc:1905] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:07.246407 26085 ts_tablet_manager.cc:1918] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:07:07.246606 26085 log.cc:1196] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/wal/wals/21ffc3de044b42f2932ecee8ff4cbc24
I20260806 13:07:07.246817 26085 ts_tablet_manager.cc:1939] T 21ffc3de044b42f2932ecee8ff4cbc24 P 591f2c06adac47ae96d94bf1ed013bf6: Deleting consensus metadata
I20260806 13:07:07.247088 25282 catalog_manager.cc:4841] TS 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237): tablet 21ffc3de044b42f2932ecee8ff4cbc24 (table test-table [id=dbfd7201c2924caeaeb2c29bb0331e71]) successfully deleted
I20260806 13:07:07.249071 25906 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ffc3de044b42f2932ecee8ff4cbc24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:07 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:38490
I20260806 13:07:07.249174 26084 tablet_replica.cc:330] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a: stopping tablet replica
I20260806 13:07:07.249225 26084 raft_consensus.cc:2229] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:07.249256 26084 raft_consensus.cc:2258] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:07.249604 26084 ts_tablet_manager.cc:1905] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:07.250140 26086 ts_tablet_manager.cc:1918] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:07:07.250191 26086 log.cc:1196] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/21ffc3de044b42f2932ecee8ff4cbc24
I20260806 13:07:07.250391 26086 ts_tablet_manager.cc:1939] T 21ffc3de044b42f2932ecee8ff4cbc24 P dba5659cf28047c090f6073d5779851b: Deleting consensus metadata
I20260806 13:07:07.250676 25283 catalog_manager.cc:4841] TS dba5659cf28047c090f6073d5779851b (127.13.224.66:39475): tablet 21ffc3de044b42f2932ecee8ff4cbc24 (table test-table [id=dbfd7201c2924caeaeb2c29bb0331e71]) successfully deleted
I20260806 13:07:07.251045 26084 ts_tablet_manager.cc:1918] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:07:07.251093 26084 log.cc:1196] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/21ffc3de044b42f2932ecee8ff4cbc24
I20260806 13:07:07.251303 26084 ts_tablet_manager.cc:1939] T 21ffc3de044b42f2932ecee8ff4cbc24 P 10e53b6569344c35be2bd90a5975616a: Deleting consensus metadata
I20260806 13:07:07.251602 25284 catalog_manager.cc:4841] TS 10e53b6569344c35be2bd90a5975616a (127.13.224.67:35123): tablet 21ffc3de044b42f2932ecee8ff4cbc24 (table test-table [id=dbfd7201c2924caeaeb2c29bb0331e71]) successfully deleted
Aug 06 13:07:07 dist-test-slave-q9h9 krb5kdc[25245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 13:07:07 dist-test-slave-q9h9 krb5kdc[25245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:07 dist-test-slave-q9h9 krb5kdc[25245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 13:07:07.280092 25371 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:07.273307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46922 (local address 127.13.224.126:35293)
0806 13:07:07.273354 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:07.273356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:07.273506 (+ 150us) server_negotiation.cc:407] Connection header received
0806 13:07:07.273566 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:07.273567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:07.273580 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:07.273631 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:07.274454 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.275250 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.276305 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.276611 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.276671 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.280027 (+ 3356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46922: BlockingRecv error: recv got EOF from 127.0.0.1:46922 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 13:07:07 dist-test-slave-q9h9 krb5kdc[25245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM
I20260806 13:07:07.290620 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.281117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46924 (local address 127.13.224.126:35293)
0806 13:07:07.281153 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:07.281154 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:07.281367 (+ 213us) server_negotiation.cc:407] Connection header received
0806 13:07:07.281414 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:07.281415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:07.281428 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:07.281486 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:07.282074 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.282816 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.284176 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.284582 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.284647 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.287782 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:07.287784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:07.287784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:07.287792 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:07.289402 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:07.289816 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:07.289817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:07.289818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:07.289880 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:07.290101 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:07.290101 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:07.290102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:07.290231 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:07.290323 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:07.290451 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 13:07:07.290529 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:07:07.292380 25301 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:07:07.292552 25301 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:07:07.294130 25499 tablet_service.cc:1467] Processing CreateTablet for tablet e00f6170063e43a5b39d0d69bfcb8b73 (DEFAULT_TABLE table=test-table [id=20bd0f778bff484886b7e17dfa4d26a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:07.294170 25906 tablet_service.cc:1467] Processing CreateTablet for tablet e00f6170063e43a5b39d0d69bfcb8b73 (DEFAULT_TABLE table=test-table [id=20bd0f778bff484886b7e17dfa4d26a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:07.294272 25499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e00f6170063e43a5b39d0d69bfcb8b73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:07.294293 25906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e00f6170063e43a5b39d0d69bfcb8b73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:07.295578 26048 tablet_bootstrap.cc:492] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6: Bootstrap starting.
I20260806 13:07:07.295625 25699 tablet_service.cc:1467] Processing CreateTablet for tablet e00f6170063e43a5b39d0d69bfcb8b73 (DEFAULT_TABLE table=test-table [id=20bd0f778bff484886b7e17dfa4d26a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:07.295734 25699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e00f6170063e43a5b39d0d69bfcb8b73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:07.295980 26048 tablet_bootstrap.cc:654] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:07.296904 26051 tablet_bootstrap.cc:492] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a: Bootstrap starting.
I20260806 13:07:07.296943 26048 tablet_bootstrap.cc:492] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6: No bootstrap required, opened a new log
I20260806 13:07:07.296986 26048 ts_tablet_manager.cc:1397] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 13:07:07.297138 26048 raft_consensus.cc:348] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.297201 26048 raft_consensus.cc:374] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:07.297220 26048 raft_consensus.cc:729] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591f2c06adac47ae96d94bf1ed013bf6, State: Initialized, Role: FOLLOWER
I20260806 13:07:07.297263 26048 consensus_queue.cc:260] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.297348 26048 ts_tablet_manager.cc:1428] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:07.297345 26051 tablet_bootstrap.cc:654] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:07.297437 26049 tablet_bootstrap.cc:492] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b: Bootstrap starting.
I20260806 13:07:07.298360 26049 tablet_bootstrap.cc:654] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:07.298972 26049 tablet_bootstrap.cc:492] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b: No bootstrap required, opened a new log
I20260806 13:07:07.299046 26049 ts_tablet_manager.cc:1397] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:07:07.299057 26051 tablet_bootstrap.cc:492] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a: No bootstrap required, opened a new log
I20260806 13:07:07.299101 26051 ts_tablet_manager.cc:1397] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:07:07.299175 26049 raft_consensus.cc:348] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.299232 26049 raft_consensus.cc:374] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:07.299248 26049 raft_consensus.cc:729] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba5659cf28047c090f6073d5779851b, State: Initialized, Role: FOLLOWER
I20260806 13:07:07.299278 26049 consensus_queue.cc:260] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.299254 26051 raft_consensus.cc:348] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.299332 26051 raft_consensus.cc:374] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:07.299346 26049 ts_tablet_manager.cc:1428] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:07.299352 26051 raft_consensus.cc:729] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10e53b6569344c35be2bd90a5975616a, State: Initialized, Role: FOLLOWER
I20260806 13:07:07.299391 26051 consensus_queue.cc:260] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.299467 26051 ts_tablet_manager.cc:1428] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:07.358083 26061 raft_consensus.cc:482] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:07:07.358158 26061 raft_consensus.cc:504] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.358314 26061 leader_election.cc:288] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237), dba5659cf28047c090f6073d5779851b (127.13.224.66:39475)
I20260806 13:07:07.358494 25528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00f6170063e43a5b39d0d69bfcb8b73" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591f2c06adac47ae96d94bf1ed013bf6" is_pre_election: true
I20260806 13:07:07.358518 25727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00f6170063e43a5b39d0d69bfcb8b73" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dba5659cf28047c090f6073d5779851b" is_pre_election: true
I20260806 13:07:07.358594 25727 raft_consensus.cc:2454] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 0.
I20260806 13:07:07.358594 25528 raft_consensus.cc:2454] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 0.
I20260806 13:07:07.358749 25828 leader_election.cc:302] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10e53b6569344c35be2bd90a5975616a, dba5659cf28047c090f6073d5779851b; no voters:
I20260806 13:07:07.358840 26061 raft_consensus.cc:2781] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:07:07.358870 26061 raft_consensus.cc:482] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:07:07.358888 26061 raft_consensus.cc:3037] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:07.359639 26061 raft_consensus.cc:504] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.359750 26061 leader_election.cc:288] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 election: Requested vote from peers 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237), dba5659cf28047c090f6073d5779851b (127.13.224.66:39475)
I20260806 13:07:07.359897 25528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00f6170063e43a5b39d0d69bfcb8b73" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591f2c06adac47ae96d94bf1ed013bf6"
I20260806 13:07:07.359918 25727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00f6170063e43a5b39d0d69bfcb8b73" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dba5659cf28047c090f6073d5779851b"
I20260806 13:07:07.359979 25528 raft_consensus.cc:3037] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:07.359982 25727 raft_consensus.cc:3037] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:07.360677 25727 raft_consensus.cc:2454] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 1.
I20260806 13:07:07.360685 25528 raft_consensus.cc:2454] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 1.
I20260806 13:07:07.360864 25828 leader_election.cc:302] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10e53b6569344c35be2bd90a5975616a, dba5659cf28047c090f6073d5779851b; no voters:
I20260806 13:07:07.360958 26061 raft_consensus.cc:2781] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:07:07.361009 26061 raft_consensus.cc:686] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [term 1 LEADER]: Becoming Leader. State: Replica: 10e53b6569344c35be2bd90a5975616a, State: Running, Role: LEADER
I20260806 13:07:07.361057 26061 consensus_queue.cc:237] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.361610 25302 catalog_manager.cc:5495] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a reported cstate change: term changed from 0 to 1, leader changed from <none> to 10e53b6569344c35be2bd90a5975616a (127.13.224.67). New cstate: current_term: 1 leader_uuid: "10e53b6569344c35be2bd90a5975616a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } health_report { overall_health: HEALTHY } } }
I20260806 13:07:07.374661 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.372297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38512 (local address 127.13.224.67:35123)
0806 13:07:07.372367 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:07.372368 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:07.372377 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:07.372399 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:07.372400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:07.372410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:07.372454 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:07:07.372556 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.373135 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.373953 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.374224 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.374284 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.374434 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:07:07.374470 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:07:07.374513 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:07.374575 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 13:07:07.374594 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 13:07:07.375277 25528 raft_consensus.cc:1261] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Refusing update from remote peer 10e53b6569344c35be2bd90a5975616a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:07.375331 25727 raft_consensus.cc:1261] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Refusing update from remote peer 10e53b6569344c35be2bd90a5975616a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:07.375475 26077 consensus_queue.cc:1035] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [LEADER]: Connected to new peer: Peer: permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:07.375590 26061 consensus_queue.cc:1035] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:07.378095 25302 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:07.378167 25302 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:07.378679 25302 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8: Sending DeleteTablet for 3 replicas of tablet e00f6170063e43a5b39d0d69bfcb8b73
I20260806 13:07:07.378898 25499 tablet_service.cc:1514] Processing DeleteTablet for tablet e00f6170063e43a5b39d0d69bfcb8b73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:07 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:39800
I20260806 13:07:07.378918 25699 tablet_service.cc:1514] Processing DeleteTablet for tablet e00f6170063e43a5b39d0d69bfcb8b73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:07 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:40416
I20260806 13:07:07.379005 26085 tablet_replica.cc:330] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6: stopping tablet replica
I20260806 13:07:07.379010 26086 tablet_replica.cc:330] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b: stopping tablet replica
I20260806 13:07:07.379011 25906 tablet_service.cc:1514] Processing DeleteTablet for tablet e00f6170063e43a5b39d0d69bfcb8b73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:07 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:38490
I20260806 13:07:07.379065 26085 raft_consensus.cc:2229] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:07.379063 26086 raft_consensus.cc:2229] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:07.379084 26084 tablet_replica.cc:330] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a: stopping tablet replica
I20260806 13:07:07.379106 26086 raft_consensus.cc:2258] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:07.379109 26085 raft_consensus.cc:2258] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:07.379128 26084 raft_consensus.cc:2229] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:07:07.379180 26084 raft_consensus.cc:2258] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:07.379387 26086 ts_tablet_manager.cc:1905] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:07.379482 26085 ts_tablet_manager.cc:1905] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:07.379444 25302 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 13:07:07.379513 26084 ts_tablet_manager.cc:1905] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 13:07:07.379567 25302 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:07:07.380717 25699 tablet_service.cc:1467] Processing CreateTablet for tablet 04339346377a4aefb5c15522e5fb43c8 (DEFAULT_TABLE table=test-table [id=7500a7c544e44ccab954fc64bdb4242f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:07.380842 25699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04339346377a4aefb5c15522e5fb43c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:07.380844 26084 ts_tablet_manager.cc:1918] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:07:07.380895 26084 log.cc:1196] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/e00f6170063e43a5b39d0d69bfcb8b73
I20260806 13:07:07.381027 26085 ts_tablet_manager.cc:1918] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:07:07.381073 26085 log.cc:1196] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-0/wal/wals/e00f6170063e43a5b39d0d69bfcb8b73
I20260806 13:07:07.381125 26084 ts_tablet_manager.cc:1939] T e00f6170063e43a5b39d0d69bfcb8b73 P 10e53b6569344c35be2bd90a5975616a: Deleting consensus metadata
I20260806 13:07:07.381331 26085 ts_tablet_manager.cc:1939] T e00f6170063e43a5b39d0d69bfcb8b73 P 591f2c06adac47ae96d94bf1ed013bf6: Deleting consensus metadata
I20260806 13:07:07.381925 26049 tablet_bootstrap.cc:492] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b: Bootstrap starting.
I20260806 13:07:07.382138 25499 tablet_service.cc:1467] Processing CreateTablet for tablet 04339346377a4aefb5c15522e5fb43c8 (DEFAULT_TABLE table=test-table [id=7500a7c544e44ccab954fc64bdb4242f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:07.382179 25284 catalog_manager.cc:4841] TS 10e53b6569344c35be2bd90a5975616a (127.13.224.67:35123): tablet e00f6170063e43a5b39d0d69bfcb8b73 (table test-table [id=20bd0f778bff484886b7e17dfa4d26a1]) successfully deleted
I20260806 13:07:07.382241 25499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04339346377a4aefb5c15522e5fb43c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:07.382251 26049 tablet_bootstrap.cc:654] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:07.382226 25906 tablet_service.cc:1467] Processing CreateTablet for tablet 04339346377a4aefb5c15522e5fb43c8 (DEFAULT_TABLE table=test-table [id=7500a7c544e44ccab954fc64bdb4242f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:07.382332 25906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04339346377a4aefb5c15522e5fb43c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:07.382688 26049 tablet_bootstrap.cc:492] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b: No bootstrap required, opened a new log
I20260806 13:07:07.382719 26049 ts_tablet_manager.cc:1397] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:07.382799 26049 raft_consensus.cc:348] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.382867 26049 raft_consensus.cc:374] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:07.382889 26049 raft_consensus.cc:729] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba5659cf28047c090f6073d5779851b, State: Initialized, Role: FOLLOWER
I20260806 13:07:07.382941 26049 consensus_queue.cc:260] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.383023 26049 ts_tablet_manager.cc:1428] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:07.383319 26048 tablet_bootstrap.cc:492] T 04339346377a4aefb5c15522e5fb43c8 P 591f2c06adac47ae96d94bf1ed013bf6: Bootstrap starting.
I20260806 13:07:07.383496 26051 tablet_bootstrap.cc:492] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a: Bootstrap starting.
I20260806 13:07:07.383682 26048 tablet_bootstrap.cc:654] T 04339346377a4aefb5c15522e5fb43c8 P 591f2c06adac47ae96d94bf1ed013bf6: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:07.383713 25282 catalog_manager.cc:4841] TS 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237): tablet e00f6170063e43a5b39d0d69bfcb8b73 (table test-table [id=20bd0f778bff484886b7e17dfa4d26a1]) successfully deleted
I20260806 13:07:07.383870 26051 tablet_bootstrap.cc:654] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:07.380844 26086 ts_tablet_manager.cc:1918] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:07:07.383973 26086 log.cc:1196] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/e00f6170063e43a5b39d0d69bfcb8b73
I20260806 13:07:07.384171 26086 ts_tablet_manager.cc:1939] T e00f6170063e43a5b39d0d69bfcb8b73 P dba5659cf28047c090f6073d5779851b: Deleting consensus metadata
I20260806 13:07:07.384481 26048 tablet_bootstrap.cc:492] T 04339346377a4aefb5c15522e5fb43c8 P 591f2c06adac47ae96d94bf1ed013bf6: No bootstrap required, opened a new log
I20260806 13:07:07.384528 26048 ts_tablet_manager.cc:1397] T 04339346377a4aefb5c15522e5fb43c8 P 591f2c06adac47ae96d94bf1ed013bf6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 13:07:07.384574 26051 tablet_bootstrap.cc:492] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a: No bootstrap required, opened a new log
I20260806 13:07:07.384616 26051 ts_tablet_manager.cc:1397] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:07.384658 26048 raft_consensus.cc:348] T 04339346377a4aefb5c15522e5fb43c8 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.384704 26048 raft_consensus.cc:374] T 04339346377a4aefb5c15522e5fb43c8 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:07.384716 26048 raft_consensus.cc:729] T 04339346377a4aefb5c15522e5fb43c8 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591f2c06adac47ae96d94bf1ed013bf6, State: Initialized, Role: FOLLOWER
I20260806 13:07:07.384723 26051 raft_consensus.cc:348] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.384754 26048 consensus_queue.cc:260] T 04339346377a4aefb5c15522e5fb43c8 P 591f2c06adac47ae96d94bf1ed013bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.384788 26051 raft_consensus.cc:374] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:07.384800 26051 raft_consensus.cc:729] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10e53b6569344c35be2bd90a5975616a, State: Initialized, Role: FOLLOWER
I20260806 13:07:07.384827 26051 consensus_queue.cc:260] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.384835 26048 ts_tablet_manager.cc:1428] T 04339346377a4aefb5c15522e5fb43c8 P 591f2c06adac47ae96d94bf1ed013bf6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:07.384887 26051 ts_tablet_manager.cc:1428] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:07.384954 25283 catalog_manager.cc:4841] TS dba5659cf28047c090f6073d5779851b (127.13.224.66:39475): tablet e00f6170063e43a5b39d0d69bfcb8b73 (table test-table [id=20bd0f778bff484886b7e17dfa4d26a1]) successfully deleted
I20260806 13:07:07.587088 26077 raft_consensus.cc:482] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:07:07.587188 26077 raft_consensus.cc:504] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.587355 26077 leader_election.cc:288] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237), dba5659cf28047c090f6073d5779851b (127.13.224.66:39475)
I20260806 13:07:07.587563 25727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04339346377a4aefb5c15522e5fb43c8" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dba5659cf28047c090f6073d5779851b" is_pre_election: true
I20260806 13:07:07.587560 25528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04339346377a4aefb5c15522e5fb43c8" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591f2c06adac47ae96d94bf1ed013bf6" is_pre_election: true
I20260806 13:07:07.587654 25727 raft_consensus.cc:2454] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 0.
I20260806 13:07:07.587656 25528 raft_consensus.cc:2454] T 04339346377a4aefb5c15522e5fb43c8 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 0.
I20260806 13:07:07.587802 25826 leader_election.cc:302] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10e53b6569344c35be2bd90a5975616a, 591f2c06adac47ae96d94bf1ed013bf6; no voters:
I20260806 13:07:07.587926 26077 raft_consensus.cc:2781] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:07:07.587980 26077 raft_consensus.cc:482] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:07:07.588001 26077 raft_consensus.cc:3037] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:07.588718 26077 raft_consensus.cc:504] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.588845 26077 leader_election.cc:288] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 election: Requested vote from peers 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237), dba5659cf28047c090f6073d5779851b (127.13.224.66:39475)
I20260806 13:07:07.588985 25528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04339346377a4aefb5c15522e5fb43c8" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591f2c06adac47ae96d94bf1ed013bf6"
I20260806 13:07:07.589051 25528 raft_consensus.cc:3037] T 04339346377a4aefb5c15522e5fb43c8 P 591f2c06adac47ae96d94bf1ed013bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:07.589048 25727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04339346377a4aefb5c15522e5fb43c8" candidate_uuid: "10e53b6569344c35be2bd90a5975616a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dba5659cf28047c090f6073d5779851b"
I20260806 13:07:07.589133 25727 raft_consensus.cc:3037] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:07.589656 25528 raft_consensus.cc:2454] T 04339346377a4aefb5c15522e5fb43c8 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 1.
I20260806 13:07:07.589699 25727 raft_consensus.cc:2454] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10e53b6569344c35be2bd90a5975616a in term 1.
I20260806 13:07:07.589833 25826 leader_election.cc:302] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10e53b6569344c35be2bd90a5975616a, 591f2c06adac47ae96d94bf1ed013bf6; no voters:
I20260806 13:07:07.589943 26077 raft_consensus.cc:2781] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:07:07.589987 26077 raft_consensus.cc:686] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [term 1 LEADER]: Becoming Leader. State: Replica: 10e53b6569344c35be2bd90a5975616a, State: Running, Role: LEADER
I20260806 13:07:07.590057 26077 consensus_queue.cc:237] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } }
I20260806 13:07:07.590613 25301 catalog_manager.cc:5495] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a reported cstate change: term changed from 0 to 1, leader changed from <none> to 10e53b6569344c35be2bd90a5975616a (127.13.224.67). New cstate: current_term: 1 leader_uuid: "10e53b6569344c35be2bd90a5975616a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10e53b6569344c35be2bd90a5975616a" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 35123 } health_report { overall_health: HEALTHY } } }
I20260806 13:07:07.614902 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.612891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:47419 (local address 127.13.224.67:35123)
0806 13:07:07.612935 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:07.612937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:07.612943 (+ 6us) server_negotiation.cc:407] Connection header received
0806 13:07:07.612961 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:07.612962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:07.612969 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:07.613010 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:07:07.613108 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.613566 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.614310 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.614499 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.614553 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.614664 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:07:07.614696 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:07:07.614744 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:07.614793 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 13:07:07.614819 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 13:07:07.614902 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.612727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:35123 (local address 127.13.224.67:47419)
0806 13:07:07.612791 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:07.612803 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:07.612840 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:07.613019 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:07.613020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:07.613021 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 13:07:07.613063 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.613064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.613628 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.613629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.614240 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:07.614242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:07.614556 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:07.614557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:07.614606 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:07.614610 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 13:07:07.614744 (+ 134us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 13:07:07.614746 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:07:07.614782 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 13:07:07.614836 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 13:07:07.616598 25727 raft_consensus.cc:1261] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Refusing update from remote peer 10e53b6569344c35be2bd90a5975616a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:07.616618 25528 raft_consensus.cc:1261] T 04339346377a4aefb5c15522e5fb43c8 P 591f2c06adac47ae96d94bf1ed013bf6 [term 1 FOLLOWER]: Refusing update from remote peer 10e53b6569344c35be2bd90a5975616a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:07.616813 26058 consensus_queue.cc:1035] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [LEADER]: Connected to new peer: Peer: permanent_uuid: "591f2c06adac47ae96d94bf1ed013bf6" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 38237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:07.616916 26077 consensus_queue.cc:1035] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dba5659cf28047c090f6073d5779851b" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:07.626333 25302 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:07.626435 25302 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:07.626999 25302 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d91c7e3dd03c49a7b11dd64c17fb0cf8: Sending DeleteTablet for 3 replicas of tablet 04339346377a4aefb5c15522e5fb43c8
I20260806 13:07:07.627233 25906 tablet_service.cc:1514] Processing DeleteTablet for tablet 04339346377a4aefb5c15522e5fb43c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:07 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:38490
I20260806 13:07:07.627259 25699 tablet_service.cc:1514] Processing DeleteTablet for tablet 04339346377a4aefb5c15522e5fb43c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:07 UTC) from {username='oryx', principal='oryx/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:40416
I20260806 13:07:07.627378 26086 tablet_replica.cc:330] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b: stopping tablet replica
I20260806 13:07:07.627457 26086 raft_consensus.cc:2229] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:07.627518 26086 raft_consensus.cc:2258] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:07.627616 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 25400
I20260806 13:07:07.628906 26084 tablet_replica.cc:330] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a: stopping tablet replica
I20260806 13:07:07.628919 26086 ts_tablet_manager.cc:1905] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:07.628978 26084 raft_consensus.cc:2229] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:07:07.629030 26084 raft_consensus.cc:2258] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:07.631134 26086 ts_tablet_manager.cc:1918] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:07:07.631198 26086 log.cc:1196] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/04339346377a4aefb5c15522e5fb43c8
I20260806 13:07:07.631403 26086 ts_tablet_manager.cc:1939] T 04339346377a4aefb5c15522e5fb43c8 P dba5659cf28047c090f6073d5779851b: Deleting consensus metadata
I20260806 13:07:07.631776 25283 catalog_manager.cc:4841] TS dba5659cf28047c090f6073d5779851b (127.13.224.66:39475): tablet 04339346377a4aefb5c15522e5fb43c8 (table test-table [id=7500a7c544e44ccab954fc64bdb4242f]) successfully deleted
I20260806 13:07:07.636791 26084 ts_tablet_manager.cc:1905] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:07.638417 26084 ts_tablet_manager.cc:1918] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 13:07:07.638477 26084 log.cc:1196] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/04339346377a4aefb5c15522e5fb43c8
I20260806 13:07:07.638659 26084 ts_tablet_manager.cc:1939] T 04339346377a4aefb5c15522e5fb43c8 P 10e53b6569344c35be2bd90a5975616a: Deleting consensus metadata
I20260806 13:07:07.638957 25284 catalog_manager.cc:4841] TS 10e53b6569344c35be2bd90a5975616a (127.13.224.67:35123): tablet 04339346377a4aefb5c15522e5fb43c8 (table test-table [id=7500a7c544e44ccab954fc64bdb4242f]) successfully deleted
W20260806 13:07:07.642367 25282 connection.cc:535] client connection to 127.13.224.65:38237 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 13:07:07.642475 25282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 13:07:07.642619 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 25610
I20260806 13:07:07.648170 26168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:07.647977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:38237 (local address 127.0.0.1:39816)
0806 13:07:07.648083 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:07.648132 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:38237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":53,"threads_started":1}
W20260806 13:07:07.649158 25282 catalog_manager.cc:4584] TS 591f2c06adac47ae96d94bf1ed013bf6 (127.13.224.65:38237): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 04339346377a4aefb5c15522e5fb43c8: Network error: Client connection negotiation failed: client connection to 127.13.224.65:38237: connect: Connection refused (error 111)
I20260806 13:07:07.653482 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 25802
I20260806 13:07:07.664844 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 25262
2026-08-06T13:07:07Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2659 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:07:07 dist-test-slave-q9h9 krb5kdc[26173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:07:07 dist-test-slave-q9h9 krb5kdc[26173](info): set up 2 sockets
Aug 06 13:07:07 dist-test-slave-q9h9 krb5kdc[26173](info): commencing operation
krb5kdc: starting...
W20260806 13:07:08.776240 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:07:08 dist-test-slave-q9h9 krb5kdc[26173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:07:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:07:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:07:08.890921 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:43241
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:46739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:43241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:08.970191 26193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:08.970386 26193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:08.970407 26193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:08.971520 26193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:07:08.971563 26193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:08.971576 26193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:08.971588 26193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:07:08.971598 26193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:07:08.973058 26193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:43241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:43241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:08.973281 26193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:08.973515 26193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:08.978164 26199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:08.978180 26200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:08.978305 26206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:08.978487 26193 server_base.cc:1034] running on GCE node
I20260806 13:07:08.978726 26193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:08.979017 26193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:08.980160 26193 hybrid_clock.cc:648] HybridClock initialized: now 1786021628980144 us; error 31 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T13:07:08Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1320 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:07:09 dist-test-slave-q9h9 krb5kdc[26211](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:07:09 dist-test-slave-q9h9 krb5kdc[26211](info): set up 2 sockets
Aug 06 13:07:09 dist-test-slave-q9h9 krb5kdc[26211](info): commencing operation
krb5kdc: starting...
W20260806 13:07:10.109500 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:07:10 dist-test-slave-q9h9 krb5kdc[26211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:07:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:07:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:07:10.221349 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:33601
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:33601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:10.302784 26224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:10.302975 26224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:10.303004 26224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:10.304134 26224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:07:10.304176 26224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:10.304195 26224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:10.304210 26224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:07:10.304227 26224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:07:10.305756 26224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:33601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:33601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:10.305996 26224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:10.306241 26224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:10.310684 26230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:10.310734 26229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:10.310684 26234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:10.311039 26224 server_base.cc:1034] running on GCE node
I20260806 13:07:10.311285 26224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:10.311549 26224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:10.312687 26224 hybrid_clock.cc:648] HybridClock initialized: now 1786021630312667 us; error 34 us; skew 500 ppm
Aug 06 13:07:10 dist-test-slave-q9h9 krb5kdc[26211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:10.319762 26224 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:07:10.320181 26224 webserver.cc:460] Webserver started at http://127.13.224.126:46185/ using document root <none> and password file <none>
I20260806 13:07:10.320392 26224 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:10.320441 26224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:10.320567 26224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:10.321573 26224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "2d5430d91c33446889ef609e6b598daa"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "e901c58ecdedf1afab9b65afbb881f01"
server_key_iv: "5e8e2dc4025a0c49aeca61d67e373a94"
server_key_version: "encryptionkey@0"
I20260806 13:07:10.321966 26224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "2d5430d91c33446889ef609e6b598daa"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "e901c58ecdedf1afab9b65afbb881f01"
server_key_iv: "5e8e2dc4025a0c49aeca61d67e373a94"
server_key_version: "encryptionkey@0"
I20260806 13:07:10.323196 26224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:10.323812 26242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:10.323980 26224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:10.324066 26224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "2d5430d91c33446889ef609e6b598daa"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "e901c58ecdedf1afab9b65afbb881f01"
server_key_iv: "5e8e2dc4025a0c49aeca61d67e373a94"
server_key_version: "encryptionkey@0"
I20260806 13:07:10.324120 26224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:10.336565 26224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:10.338181 26224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:10.338316 26224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:10.341995 26224 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:33601
I20260806 13:07:10.342044 26318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:33601 every 8 connection(s)
I20260806 13:07:10.342363 26224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:07:10.343143 26320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:10.345247 26320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa: Bootstrap starting.
I20260806 13:07:10.345752 26320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:10.345791 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 26224
I20260806 13:07:10.345898 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:07:10.346066 26320 log.cc:824] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:10.346089 14209 external_mini_cluster.cc:1428] Setting key c32befa4e7c7db8581b14f8591a2352b
I20260806 13:07:10.346777 26320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa: No bootstrap required, opened a new log
I20260806 13:07:10.348737 26320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5430d91c33446889ef609e6b598daa" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33601 } }
I20260806 13:07:10.348869 26320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:10.348902 26320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5430d91c33446889ef609e6b598daa, State: Initialized, Role: FOLLOWER
I20260806 13:07:10.349051 26320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5430d91c33446889ef609e6b598daa" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33601 } }
I20260806 13:07:10.349131 26320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:07:10.349162 26320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:07:10.349218 26320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:10.349715 26320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5430d91c33446889ef609e6b598daa" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33601 } }
I20260806 13:07:10.349808 26320 leader_election.cc:302] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d5430d91c33446889ef609e6b598daa; no voters:
I20260806 13:07:10.349962 26320 leader_election.cc:288] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:07:10.349998 26330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:07:10.350126 26330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 1 LEADER]: Becoming Leader. State: Replica: 2d5430d91c33446889ef609e6b598daa, State: Running, Role: LEADER
I20260806 13:07:10.350246 26320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:07:10.350267 26330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5430d91c33446889ef609e6b598daa" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33601 } }
I20260806 13:07:10.350647 26330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d5430d91c33446889ef609e6b598daa. Latest consensus state: current_term: 1 leader_uuid: "2d5430d91c33446889ef609e6b598daa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5430d91c33446889ef609e6b598daa" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33601 } } }
I20260806 13:07:10.350698 26331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d5430d91c33446889ef609e6b598daa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5430d91c33446889ef609e6b598daa" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33601 } } }
I20260806 13:07:10.350818 26331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:10.350803 26330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:10.351512 26336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:07:10.352042 26336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 13:07:10 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:07:10.353969 26336 catalog_manager.cc:1347] Generated new cluster ID: 5b717e661f9240dd860fd4ef7d26298e
I20260806 13:07:10.354010 26336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:07:10.356812 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:10.346566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.13.224.126:33601)
0806 13:07:10.346720 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:10.346729 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:10.346761 (+ 32us) server_negotiation.cc:407] Connection header received
0806 13:07:10.347187 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:10.347194 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:10.347240 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:10.347369 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:10.347999 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.348859 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.349699 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.350284 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.350666 (+ 382us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:10.353344 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:10.353348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:10.353348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:10.353360 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:10.355276 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:10.355634 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:10.355637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:10.355638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:10.355753 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:10.355924 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:10.355924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:10.355925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:10.356228 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:10.356379 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:10.356456 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 13:07:10.356589 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260806 13:07:10.362846 26336 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:07:10.363504 26336 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:07:10.368137 26336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa: Generated new TSK 0
I20260806 13:07:10.368299 26336 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:07:10.394644 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33601
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:10.475544 26359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:10.475739 26359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:10.475762 26359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:10.476893 26359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:10.476933 26359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:10.476958 26359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:07:10.478399 26359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:10.478624 26359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:10.478868 26359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:10.483592 26369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:10.483588 26371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:10.483587 26367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:10.483865 26359 server_base.cc:1034] running on GCE node
I20260806 13:07:10.484081 26359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:10.484313 26359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:10.485452 26359 hybrid_clock.cc:648] HybridClock initialized: now 1786021630485438 us; error 27 us; skew 500 ppm
Aug 06 13:07:10 dist-test-slave-q9h9 krb5kdc[26211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:10.487819 26359 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:07:10.488209 26359 webserver.cc:460] Webserver started at http://127.13.224.65:41031/ using document root <none> and password file <none>
I20260806 13:07:10.488422 26359 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:10.488471 26359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:10.488595 26359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:10.489566 26359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "f377552ad1b043699a996525da697ed6"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "fdc616025d183a9010bf55cfedbdff2c"
server_key_iv: "88c26e3b3a8d6bdb84cf07f7182a3ce9"
server_key_version: "encryptionkey@0"
I20260806 13:07:10.489892 26359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "f377552ad1b043699a996525da697ed6"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "fdc616025d183a9010bf55cfedbdff2c"
server_key_iv: "88c26e3b3a8d6bdb84cf07f7182a3ce9"
server_key_version: "encryptionkey@0"
I20260806 13:07:10.491097 26359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:07:10.491719 26380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:10.491869 26359 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:10.491950 26359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "f377552ad1b043699a996525da697ed6"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "fdc616025d183a9010bf55cfedbdff2c"
server_key_iv: "88c26e3b3a8d6bdb84cf07f7182a3ce9"
server_key_version: "encryptionkey@0"
I20260806 13:07:10.492009 26359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:10.512132 26359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:10.513767 26359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:10.513893 26359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:10.514125 26359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:10.514420 26359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:10.514453 26359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:10.514482 26359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:10.514504 26359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:10.520085 26359 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:45707
I20260806 13:07:10.520148 26553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:45707 every 8 connection(s)
I20260806 13:07:10.520435 26359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
Aug 06 13:07:10 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:07:10.528769 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:10.521089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:40311 (local address 127.13.224.126:33601)
0806 13:07:10.521133 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:10.521135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:10.521320 (+ 185us) server_negotiation.cc:407] Connection header received
0806 13:07:10.521805 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:10.521806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:10.521816 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:10.521867 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:10.522839 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.523331 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.524241 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.524435 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.524476 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:10.526407 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:10.526409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:10.526409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:10.526418 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:10.527601 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:10.527954 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:10.527955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:10.527955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:10.528002 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:10.528231 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:10.528233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:10.528234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:10.528364 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:10.528430 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:10.528625 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 13:07:10.528695 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:07:10.528904 26557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:10.521052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:40311)
0806 13:07:10.521204 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:10.521243 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:10.521732 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:10.521921 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:10.521924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:10.522533 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:10.522761 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:10.522767 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.523395 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.523396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:10.524175 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:10.524178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.524487 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.524488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:10.524590 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:10.524994 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:10.525002 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:10.526310 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:10.527669 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:10.527676 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:10.527677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:10.527899 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:10.528049 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:10.528049 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:10.528050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:10.528147 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:10.528423 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:10.528426 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:10.528530 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 13:07:10.528603 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 13:07:10.528717 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260806 13:07:10.529412 26554 heartbeater.cc:347] Connected to a master server at 127.13.224.126:33601
I20260806 13:07:10.529469 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 26359
I20260806 13:07:10.529507 26554 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:10.529567 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:07:10.529695 26554 heartbeater.cc:511] Master 127.13.224.126:33601 requested a full tablet report, sending...
I20260806 13:07:10.529753 14209 external_mini_cluster.cc:1428] Setting key d7ec3c28773210ba3a957fe5c797d506
I20260806 13:07:10.530259 26266 ts_manager.cc:194] Registered new tserver with Master: f377552ad1b043699a996525da697ed6 (127.13.224.65:45707)
I20260806 13:07:10.531072 26266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:40311
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:07:10.553859 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33601
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:10.633293 26560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:10.633482 26560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:10.633512 26560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:10.634686 26560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:10.634727 26560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:10.634761 26560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:07:10.636168 26560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:10.636379 26560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:10.636577 26560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:10.641194 26568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:10.641225 26572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:10.641260 26570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:10.641836 26560 server_base.cc:1034] running on GCE node
I20260806 13:07:10.642076 26560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:10.642297 26560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:10.643438 26560 hybrid_clock.cc:648] HybridClock initialized: now 1786021630643424 us; error 34 us; skew 500 ppm
Aug 06 13:07:10 dist-test-slave-q9h9 krb5kdc[26211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:10.645700 26560 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:07:10.646163 26560 webserver.cc:460] Webserver started at http://127.13.224.66:34077/ using document root <none> and password file <none>
I20260806 13:07:10.646364 26560 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:10.646411 26560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:10.646535 26560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:10.647564 26560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "50a1693522204fe6961f0fe823e149a7"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "c31db05b31634a1f7671b89e91844cf5"
server_key_iv: "89c3e6e9565e6ca360ea5ed102405f6b"
server_key_version: "encryptionkey@0"
I20260806 13:07:10.647892 26560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "50a1693522204fe6961f0fe823e149a7"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "c31db05b31634a1f7671b89e91844cf5"
server_key_iv: "89c3e6e9565e6ca360ea5ed102405f6b"
server_key_version: "encryptionkey@0"
I20260806 13:07:10.649075 26560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:07:10.649739 26581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:10.649876 26560 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:10.649957 26560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "50a1693522204fe6961f0fe823e149a7"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "c31db05b31634a1f7671b89e91844cf5"
server_key_iv: "89c3e6e9565e6ca360ea5ed102405f6b"
server_key_version: "encryptionkey@0"
I20260806 13:07:10.650048 26560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:10.671444 26560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:10.673017 26560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:10.673136 26560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:10.673310 26560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:10.673578 26560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:10.673610 26560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:10.673637 26560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:10.673653 26560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:10.679234 26560 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:41175
I20260806 13:07:10.679291 26739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:41175 every 8 connection(s)
I20260806 13:07:10.679553 26560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
Aug 06 13:07:10 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:07:10.688321 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 26560
I20260806 13:07:10.688455 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:07:10.688683 14209 external_mini_cluster.cc:1428] Setting key e9379a711b4960355c5b92b4bbae66df
I20260806 13:07:10.690825 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:10.680149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:57849 (local address 127.13.224.126:33601)
0806 13:07:10.680201 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:10.680204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:10.680387 (+ 183us) server_negotiation.cc:407] Connection header received
0806 13:07:10.680824 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:10.680825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:10.680836 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:10.680876 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:10.681756 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.682333 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.683231 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.683413 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.683458 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:10.687878 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:10.687880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:10.687880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:10.687889 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:10.689486 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:10.689922 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:10.689922 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:10.689923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:10.689970 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:10.690256 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:10.690257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:10.690257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:10.690359 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:10.690412 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:10.690703 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 13:07:10.690762 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 13:07:10.691038 26742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:10.680105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:57849)
0806 13:07:10.680268 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:10.680308 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:10.680735 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:10.680937 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:10.680940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:10.681492 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:10.681665 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:10.681670 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.682403 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.682403 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:10.683158 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:10.683160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.683464 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.683465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:10.683556 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:10.686394 (+ 2838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:10.686403 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:10.687773 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:10.689561 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:10.689568 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:10.689575 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:10.689858 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:10.690040 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:10.690041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:10.690041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:10.690174 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:10.690415 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:10.690421 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:10.690571 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 13:07:10.690678 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 13:07:10.690831 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
I20260806 13:07:10.691501 26740 heartbeater.cc:347] Connected to a master server at 127.13.224.126:33601
I20260806 13:07:10.691582 26740 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:10.691743 26740 heartbeater.cc:511] Master 127.13.224.126:33601 requested a full tablet report, sending...
I20260806 13:07:10.692075 26266 ts_manager.cc:194] Registered new tserver with Master: 50a1693522204fe6961f0fe823e149a7 (127.13.224.66:41175)
I20260806 13:07:10.692497 26266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:57849
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:07:10.712612 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33601
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:10.792989 26745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:10.793185 26745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:10.793218 26745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:10.794436 26745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:10.794479 26745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:10.794513 26745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:07:10.795982 26745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:33601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:10.796206 26745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:10.796402 26745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:10.801110 26753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:10.801133 26755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:10.801133 26752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:10.801295 26745 server_base.cc:1034] running on GCE node
I20260806 13:07:10.801517 26745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:10.801753 26745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:10.802889 26745 hybrid_clock.cc:648] HybridClock initialized: now 1786021630802874 us; error 27 us; skew 500 ppm
Aug 06 13:07:10 dist-test-slave-q9h9 krb5kdc[26211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:10.805320 26745 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:07:10.805748 26745 webserver.cc:460] Webserver started at http://127.13.224.67:43491/ using document root <none> and password file <none>
I20260806 13:07:10.805956 26745 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:10.806005 26745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:10.806159 26745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:10.807091 26745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "6e49cd9bc94249209bce4d3de03c7269"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "bd5df4f35c5c54bcb88dd8c46b397c5d"
server_key_iv: "a89e4795d75f49016618af9c76900246"
server_key_version: "encryptionkey@0"
I20260806 13:07:10.807368 26745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "6e49cd9bc94249209bce4d3de03c7269"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "bd5df4f35c5c54bcb88dd8c46b397c5d"
server_key_iv: "a89e4795d75f49016618af9c76900246"
server_key_version: "encryptionkey@0"
I20260806 13:07:10.808566 26745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:07:10.809202 26763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:10.809356 26745 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:10.809439 26745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "6e49cd9bc94249209bce4d3de03c7269"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "bd5df4f35c5c54bcb88dd8c46b397c5d"
server_key_iv: "a89e4795d75f49016618af9c76900246"
server_key_version: "encryptionkey@0"
I20260806 13:07:10.809497 26745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:10.826344 26745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:10.828147 26745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:10.828274 26745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:10.828464 26745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:10.828740 26745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:10.828775 26745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:10.828802 26745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:10.828826 26745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:10.834415 26745 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:44391
I20260806 13:07:10.834474 26919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:44391 every 8 connection(s)
I20260806 13:07:10.834757 26745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:07:10.836884 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 26745
I20260806 13:07:10.836978 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:07:10.837186 14209 external_mini_cluster.cc:1428] Setting key 9777ded976767e9692a7f2ee41135677
Aug 06 13:07:10 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021630, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:07:10.843505 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:10.835374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:59197 (local address 127.13.224.126:33601)
0806 13:07:10.835437 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:10.835439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:10.835639 (+ 200us) server_negotiation.cc:407] Connection header received
0806 13:07:10.836071 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:10.836072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:10.836081 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:10.836132 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:10.837059 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.837648 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.838708 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.838899 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.838955 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:10.840962 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:10.840963 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:10.840964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:10.840971 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:10.842248 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:10.842610 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:10.842611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:10.842611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:10.842666 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:10.842893 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:10.842894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:10.842895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:10.843003 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:10.843058 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:10.843334 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 13:07:10.843414 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 13:07:10.843657 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:10.835356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:59197)
0806 13:07:10.835522 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:10.835561 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:10.835999 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:10.836181 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:10.836184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:10.836747 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:10.836928 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:10.836934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.837736 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.837737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:10.838626 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:10.838629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:10.838968 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:10.838969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:10.839095 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:10.839523 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:10.839530 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:10.840856 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:10.842317 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:10.842324 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:10.842326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:10.842543 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:10.842731 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:10.842732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:10.842732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:10.842838 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:10.843058 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:10.843065 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:10.843212 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 13:07:10.843302 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 13:07:10.843452 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260806 13:07:10.844122 26920 heartbeater.cc:347] Connected to a master server at 127.13.224.126:33601
I20260806 13:07:10.844211 26920 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:10.844384 26920 heartbeater.cc:511] Master 127.13.224.126:33601 requested a full tablet report, sending...
I20260806 13:07:10.844794 26266 ts_manager.cc:194] Registered new tserver with Master: 6e49cd9bc94249209bce4d3de03c7269 (127.13.224.67:44391)
I20260806 13:07:10.845173 26266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:59197
I20260806 13:07:10.848834 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab.
Entry for principal oryx/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab.
WARNING: no policy specified for oryx/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab.
Entry for principal oryx/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab.
WARNING: no policy specified for oryx/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab.
Entry for principal oryx/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab.
I20260806 13:07:10.909634 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 26359
I20260806 13:07:10.921731 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 26560
I20260806 13:07:10.929797 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 26745
I20260806 13:07:10.947942 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 26224
I20260806 13:07:10.959931 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:33601
--webserver_interface=127.13.224.126
--webserver_port=46185
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:33601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:11.040002 26929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:11.040197 26929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:11.040227 26929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:11.041330 26929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:07:11.041370 26929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:11.041389 26929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:11.041404 26929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:07:11.041422 26929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:07:11.042924 26929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:33601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:33601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=46185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:11.043164 26929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:11.043365 26929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:11.047875 26939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:11.047842 26941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:11.047889 26929 server_base.cc:1034] running on GCE node
W20260806 13:07:11.047852 26938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:11.048290 26929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:11.048570 26929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:11.049716 26929 hybrid_clock.cc:648] HybridClock initialized: now 1786021631049696 us; error 32 us; skew 500 ppm
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021631, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:11.052018 26929 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:07:11.052431 26929 webserver.cc:460] Webserver started at http://127.13.224.126:46185/ using document root <none> and password file <none>
I20260806 13:07:11.052634 26929 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:11.052691 26929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:11.053902 26929 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:07:11.054550 26948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:11.054750 26929 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:11.054852 26929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "2d5430d91c33446889ef609e6b598daa"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "e901c58ecdedf1afab9b65afbb881f01"
server_key_iv: "5e8e2dc4025a0c49aeca61d67e373a94"
server_key_version: "encryptionkey@0"
I20260806 13:07:11.055174 26929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:11.074352 26929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:11.076005 26929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:11.076123 26929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:11.079934 26929 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:33601
I20260806 13:07:11.079967 27029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:33601 every 8 connection(s)
I20260806 13:07:11.080291 26929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:07:11.081672 27030 sys_catalog.cc:263] Verifying existing consensus state
I20260806 13:07:11.082326 27030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa: Bootstrap starting.
I20260806 13:07:11.084573 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 26929
I20260806 13:07:11.084910 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:45707
--local_ip_for_outbound_sockets=127.13.224.65
--tserver_master_addrs=127.13.224.126:33601
--webserver_port=41031
--webserver_interface=127.13.224.65
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.224.65
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:07:11.090889 27030 log.cc:824] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:11.092397 27030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:07:11.092640 27030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa: Bootstrap complete.
I20260806 13:07:11.094730 27030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5430d91c33446889ef609e6b598daa" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33601 } }
I20260806 13:07:11.094967 27030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5430d91c33446889ef609e6b598daa, State: Initialized, Role: FOLLOWER
I20260806 13:07:11.095155 27030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5430d91c33446889ef609e6b598daa" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33601 } }
I20260806 13:07:11.095232 27030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:07:11.095283 27030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:07:11.095329 27030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 1 FOLLOWER]: Advancing to term 2
I20260806 13:07:11.096251 27030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5430d91c33446889ef609e6b598daa" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33601 } }
I20260806 13:07:11.096342 27030 leader_election.cc:302] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d5430d91c33446889ef609e6b598daa; no voters:
I20260806 13:07:11.096494 27030 leader_election.cc:288] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 13:07:11.096558 27037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 2 FOLLOWER]: Leader election won for term 2
I20260806 13:07:11.096731 27030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:07:11.096764 27037 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [term 2 LEADER]: Becoming Leader. State: Replica: 2d5430d91c33446889ef609e6b598daa, State: Running, Role: LEADER
I20260806 13:07:11.096911 27037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5430d91c33446889ef609e6b598daa" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33601 } }
I20260806 13:07:11.097187 27039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2d5430d91c33446889ef609e6b598daa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5430d91c33446889ef609e6b598daa" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33601 } } }
I20260806 13:07:11.097251 27039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:11.097638 27040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d5430d91c33446889ef609e6b598daa. Latest consensus state: current_term: 2 leader_uuid: "2d5430d91c33446889ef609e6b598daa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5430d91c33446889ef609e6b598daa" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 33601 } } }
I20260806 13:07:11.097708 27040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:11.097772 27047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:07:11.098315 27047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:07:11.098457 27047 catalog_manager.cc:1259] Loaded cluster ID: 5b717e661f9240dd860fd4ef7d26298e
I20260806 13:07:11.098493 27047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:07:11.099750 27047 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:07:11.099912 27047 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2d5430d91c33446889ef609e6b598daa: Loaded TSK: 0
I20260806 13:07:11.100114 27047 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 13:07:11.167460 27033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:11.167639 27033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:11.167660 27033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:11.168764 27033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:11.168802 27033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:11.168828 27033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:07:11.170264 27033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.65.keytab
--principal=oryx/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:45707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=41031
--tserver_master_addrs=127.13.224.126:33601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:11.170485 27033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:11.170694 27033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:11.175523 27071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:11.175537 27073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:11.175544 27068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:11.175817 27033 server_base.cc:1034] running on GCE node
I20260806 13:07:11.176019 27033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:11.176288 27033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:11.177465 27033 hybrid_clock.cc:648] HybridClock initialized: now 1786021631177445 us; error 32 us; skew 500 ppm
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021631, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:11.180428 27033 init.cc:377] Logged in from keytab as oryx/127.13.224.65@KRBTEST.COM (short username oryx)
I20260806 13:07:11.180943 27033 webserver.cc:460] Webserver started at http://127.13.224.65:41031/ using document root <none> and password file <none>
I20260806 13:07:11.181210 27033 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:11.181288 27033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:11.182966 27033 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:11.183609 27082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:11.183768 27033 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:11.183861 27033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "f377552ad1b043699a996525da697ed6"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "fdc616025d183a9010bf55cfedbdff2c"
server_key_iv: "88c26e3b3a8d6bdb84cf07f7182a3ce9"
server_key_version: "encryptionkey@0"
I20260806 13:07:11.184263 27033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:11.201548 27033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:11.203871 27033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:11.204020 27033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:11.204267 27033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:11.204619 27033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:11.204681 27033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:11.204711 27033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:11.204728 27033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:11.212172 27235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:45707 every 8 connection(s)
I20260806 13:07:11.212227 27033 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:45707
I20260806 13:07:11.212690 27033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:11.220546 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 27033
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:11.220863 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:41175
--local_ip_for_outbound_sockets=127.13.224.66
--tserver_master_addrs=127.13.224.126:33601
--webserver_port=34077
--webserver_interface=127.13.224.66
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.224.66
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:07:11.221691 27239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:11.213448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:43069)
0806 13:07:11.213638 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:11.213683 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:11.214299 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:11.214716 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:11.214720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:11.215410 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:11.215668 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.215674 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.216760 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.216761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.217976 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.217979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.218585 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.218586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.218740 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.219185 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:11.219195 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:11.221499 (+ 2304us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
W20260806 13:07:11.221920 27236 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:33601 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.224.126:33601: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
W20260806 13:07:11.222190 27242 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:11.213560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:43069 (local address 127.13.224.126:33601)
0806 13:07:11.213740 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:11.213748 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:11.213774 (+ 26us) server_negotiation.cc:407] Connection header received
0806 13:07:11.214504 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:11.214510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:11.214553 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:11.214665 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:11.215813 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.216668 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.218115 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.218506 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.218581 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.221710 (+ 3129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:43069: BlockingRecv error: recv got EOF from 127.13.224.65:43069 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:11.227339 27239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:11.222139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:52453)
0806 13:07:11.222438 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:11.222449 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:11.222492 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:11.222630 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:11.222631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:11.222785 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:11.222834 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.222835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.223665 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.223666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.224444 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.224447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.224878 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.224879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.224941 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.225339 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:11.225344 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:11.227271 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":61,"threads_started":1}
W20260806 13:07:11.227723 27242 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:11.222302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:52453 (local address 127.13.224.126:33601)
0806 13:07:11.222474 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:11.222476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:11.222483 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:11.222541 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:11.222542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:11.222553 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:11.222902 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:11.222932 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.223569 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.224536 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.224803 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.224879 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.227366 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:52453: BlockingRecv error: recv got EOF from 127.13.224.65:52453 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":75,"threads_started":1}
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:11.232664 27243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:11.227560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:33835)
0806 13:07:11.227646 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:11.227655 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:11.227697 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:11.227940 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:11.227941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:11.228108 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:11.228158 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.228160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.229025 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.229026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.229742 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.229745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.230218 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.230220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.230277 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.230712 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:11.230716 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:11.232611 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 13:07:11.232776 27244 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:11.227781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:33835 (local address 127.13.224.126:33601)
0806 13:07:11.227824 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:11.227826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:11.227833 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:11.227853 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:11.227853 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:11.227862 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:11.227928 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:11.228238 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.228936 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.229830 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.230131 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.230199 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.232707 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:33835: BlockingRecv error: recv got EOF from 127.13.224.65:33835 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 13:07:11.232887 27236 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 13:07:11.305199 27245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:11.305377 27245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:11.305397 27245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:11.306566 27245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:11.306607 27245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:11.306633 27245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:07:11.308061 27245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.66.keytab
--principal=oryx/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:41175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=34077
--tserver_master_addrs=127.13.224.126:33601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:11.308252 27245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:11.308419 27245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:11.313030 27256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:11.313054 27250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:11.313035 27252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:11.313117 27245 server_base.cc:1034] running on GCE node
I20260806 13:07:11.313405 27245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:11.313629 27245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:11.314762 27245 hybrid_clock.cc:648] HybridClock initialized: now 1786021631314747 us; error 29 us; skew 500 ppm
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021631, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:11.317055 27245 init.cc:377] Logged in from keytab as oryx/127.13.224.66@KRBTEST.COM (short username oryx)
I20260806 13:07:11.317474 27245 webserver.cc:460] Webserver started at http://127.13.224.66:34077/ using document root <none> and password file <none>
I20260806 13:07:11.317685 27245 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:11.317747 27245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:11.318995 27245 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:11.319507 27268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:11.319649 27245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:11.319728 27245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "50a1693522204fe6961f0fe823e149a7"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "c31db05b31634a1f7671b89e91844cf5"
server_key_iv: "89c3e6e9565e6ca360ea5ed102405f6b"
server_key_version: "encryptionkey@0"
I20260806 13:07:11.320014 27245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:11.327216 27245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:11.329071 27245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:11.329202 27245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:11.329383 27245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:11.329640 27245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:11.329674 27245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:11.329698 27245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:11.329722 27245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:11.335664 27245 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:41175
I20260806 13:07:11.335719 27432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:41175 every 8 connection(s)
I20260806 13:07:11.336007 27245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:07:11.336700 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 27245
I20260806 13:07:11.336969 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:44391
--local_ip_for_outbound_sockets=127.13.224.67
--tserver_master_addrs=127.13.224.126:33601
--webserver_port=43491
--webserver_interface=127.13.224.67
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.224.67
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 13:07:11.343253 27244 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:11.336711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:54031 (local address 127.13.224.126:33601)
0806 13:07:11.336775 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:11.336777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:11.336958 (+ 181us) server_negotiation.cc:407] Connection header received
0806 13:07:11.337496 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:11.337498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:11.337512 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:11.337561 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:11.338483 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.338977 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.339919 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.340123 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.340177 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.343208 (+ 3031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:54031: BlockingRecv error: recv got EOF from 127.13.224.66:54031 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 13:07:11.343322 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:11.336703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:54031)
0806 13:07:11.336848 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:11.336886 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:11.337396 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:11.337628 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:11.337633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:11.338210 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:11.338391 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.338396 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.339058 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.339059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.339833 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.339835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.340192 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.340193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.340302 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.340758 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:11.340768 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:11.343128 (+ 2360us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
W20260806 13:07:11.343544 27433 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:33601 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.224.126:33601: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:11.347620 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:11.343766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:41593)
0806 13:07:11.343857 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:11.343866 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:11.343908 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:11.344013 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:11.344013 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:11.344137 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:11.344179 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.344181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.344754 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.344755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.345248 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.345249 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.345554 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.345556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.345617 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.345954 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:11.345959 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:11.347556 (+ 1597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":77}
W20260806 13:07:11.347679 27244 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:11.343819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:41593 (local address 127.13.224.126:33601)
0806 13:07:11.343855 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:11.343857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:11.343896 (+ 39us) server_negotiation.cc:407] Connection header received
0806 13:07:11.343962 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:11.343962 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:11.343971 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:11.344011 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:11.344245 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.344690 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.345316 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.345500 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.345550 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.347632 (+ 2082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:41593: BlockingRecv error: recv got EOF from 127.13.224.66:41593 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:11.351576 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:11.347920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:54415)
0806 13:07:11.347960 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:11.347970 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:11.348005 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:11.348107 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:11.348108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:11.348228 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:11.348253 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.348254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.348818 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.348819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.349297 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.349298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.349598 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.349599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.349647 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.349995 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:11.349998 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:11.351523 (+ 1525us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 13:07:11.351704 27433 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 13:07:11.351824 27244 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:11.347985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:54415 (local address 127.13.224.126:33601)
0806 13:07:11.348019 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:11.348021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:11.348028 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:11.348047 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:11.348048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:11.348056 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:11.348097 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:11.348317 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.348754 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.349365 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.349544 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.349594 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.351606 (+ 2012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:54415: BlockingRecv error: recv got EOF from 127.13.224.66:54415 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 13:07:11.418169 27437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:11.418344 27437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:11.418365 27437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:11.419471 27437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:11.419508 27437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:11.419538 27437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:07:11.420930 27437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/krb5kdc/oryx_127.13.224.67.keytab
--principal=oryx/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:44391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=43491
--tserver_master_addrs=127.13.224.126:33601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:11.421157 27437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:11.421351 27437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:11.426200 27447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:11.426208 27437 server_base.cc:1034] running on GCE node
W20260806 13:07:11.426170 27444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:11.426191 27443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:11.426540 27437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:11.426731 27437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:11.427862 27437 hybrid_clock.cc:648] HybridClock initialized: now 1786021631427849 us; error 30 us; skew 500 ppm
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021631, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:11.430253 27437 init.cc:377] Logged in from keytab as oryx/127.13.224.67@KRBTEST.COM (short username oryx)
I20260806 13:07:11.430648 27437 webserver.cc:460] Webserver started at http://127.13.224.67:43491/ using document root <none> and password file <none>
I20260806 13:07:11.430871 27437 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:11.430933 27437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:11.432360 27437 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:11.432912 27464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:11.433092 27437 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:11.433179 27437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "6e49cd9bc94249209bce4d3de03c7269"
format_stamp: "Formatted at 2026-08-06 13:07:10 on dist-test-slave-q9h9"
server_key: "bd5df4f35c5c54bcb88dd8c46b397c5d"
server_key_iv: "a89e4795d75f49016618af9c76900246"
server_key_version: "encryptionkey@0"
I20260806 13:07:11.433490 27437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:11.446602 27437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:11.448166 27437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:11.448300 27437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:11.448480 27437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:11.448745 27437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:11.448777 27437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:11.448803 27437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:11.448820 27437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:11.454289 27437 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:44391
I20260806 13:07:11.454326 27618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:44391 every 8 connection(s)
I20260806 13:07:11.454617 27437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 13:07:11.461171 27244 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:11.455274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:40221 (local address 127.13.224.126:33601)
0806 13:07:11.455347 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:11.455349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:11.455482 (+ 133us) server_negotiation.cc:407] Connection header received
0806 13:07:11.455955 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:11.455956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:11.455966 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:11.456014 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:11.456859 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.457362 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.458334 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.458512 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.458562 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.461116 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:40221: BlockingRecv error: recv got EOF from 127.13.224.67:40221 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 13:07:11.461202 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:11.455237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:40221)
0806 13:07:11.455378 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:11.455411 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:11.455868 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:11.456076 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:11.456078 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:11.456596 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:11.456793 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.456797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.457428 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.457429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.458275 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.458278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.458580 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.458581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.458677 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.459090 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:11.459098 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:11.461021 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
W20260806 13:07:11.461391 27619 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:33601 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.224.126:33601: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
I20260806 13:07:11.461953 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 27437
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:11.465966 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:11.461582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:43395)
0806 13:07:11.461623 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:11.461634 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:11.461676 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:11.461887 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:11.461888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:11.461989 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:11.462060 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.462062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.462814 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.462815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.463492 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.463494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.463903 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.463903 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.463960 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.464265 (+ 305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:11.464269 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:11.465912 (+ 1643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 13:07:11.466061 27244 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:11.461734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:43395 (local address 127.13.224.126:33601)
0806 13:07:11.461786 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:11.461788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:11.461797 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:11.461817 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:11.461818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:11.461827 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:11.461883 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:11.462126 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.462743 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.463555 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.463829 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.463890 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.465993 (+ 2103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:43395: BlockingRecv error: recv got EOF from 127.13.224.67:43395 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:11.469014 27242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:11.462350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60540 (local address 127.13.224.126:33601)
0806 13:07:11.462389 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:11.462391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:11.462518 (+ 127us) server_negotiation.cc:407] Connection header received
0806 13:07:11.462572 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:11.462573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:11.462585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:11.462634 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:11.463291 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.463844 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.464817 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.465093 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.465163 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.466323 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:11.466327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:11.466328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:11.466339 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:11.467771 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:11.468115 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:11.468118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:11.468118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:11.468214 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:11.468385 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:11.468386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:11.468386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:11.468612 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:11.468749 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:11.468815 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 13:07:11.468933 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 13:07:11 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:11.469950 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:11.466203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:57877)
0806 13:07:11.466306 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:11.466315 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:11.466357 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:11.466463 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:11.466463 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:11.466570 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:11.466594 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.466596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.467136 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.467137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.467600 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:11.467601 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.467924 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.467925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:11.467972 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.468272 (+ 300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:11.468276 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:11.469897 (+ 1621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":68,"threads_started":1}
W20260806 13:07:11.470001 27244 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:11.466264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:57877 (local address 127.13.224.126:33601)
0806 13:07:11.466298 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:11.466300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:11.466358 (+ 58us) server_negotiation.cc:407] Connection header received
0806 13:07:11.466409 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:11.466410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:11.466418 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:11.466461 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:11.466653 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.467074 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.467664 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:11.467859 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:11.467916 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:11.469962 (+ 2046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:57877: BlockingRecv error: recv got EOF from 127.13.224.67:57877 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 13:07:11.470139 27619 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 13:07:12 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:12 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:12.237769 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:12.233218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:44617)
0806 13:07:12.233352 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:12.233364 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:12.233431 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:12.233555 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:12.233555 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:12.233711 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:12.233761 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:12.233763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:12.234406 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:12.234408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:12.234955 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:12.234957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:12.235274 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:12.235275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:12.235330 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:12.235801 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:12.235805 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:12.237700 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
W20260806 13:07:12.237867 27627 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:12.233308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:44617 (local address 127.13.224.126:33601)
0806 13:07:12.233448 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:12.233449 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:12.233469 (+ 20us) server_negotiation.cc:407] Connection header received
0806 13:07:12.233488 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:12.233489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:12.233498 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:12.233543 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:12.233812 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:12.234320 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:12.235020 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:12.235197 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:12.235239 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:12.237795 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:44617: BlockingRecv error: recv got EOF from 127.13.224.65:44617 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
Aug 06 13:07:12 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:12 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:12.356418 27629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:12.352052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:57507)
0806 13:07:12.352164 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:12.352174 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:12.352230 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:12.352359 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:12.352359 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:12.352514 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:12.352563 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:12.352564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:12.353150 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:12.353150 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:12.353755 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:12.353759 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:12.354199 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:12.354200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:12.354266 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:12.354635 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:12.354638 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:12.356344 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":54,"threads_started":1}
W20260806 13:07:12.356496 27627 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:12.352113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:57507 (local address 127.13.224.126:33601)
0806 13:07:12.352191 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:12.352192 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:12.352215 (+ 23us) server_negotiation.cc:407] Connection header received
0806 13:07:12.352288 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:12.352289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:12.352301 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:12.352346 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:12.352609 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:12.353098 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:12.353833 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:12.354131 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:12.354189 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:12.356435 (+ 2246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:57507: BlockingRecv error: recv got EOF from 127.13.224.66:57507 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 13:07:12 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:12 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:12.474998 27630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:12.470470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:40691)
0806 13:07:12.470589 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:12.470601 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:12.470675 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:12.470816 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:12.470817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:12.471022 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:12.471083 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:12.471086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:12.472013 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:12.472014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:12.472596 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:12.472599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:12.472916 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:12.472917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:12.472954 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:12.473336 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:12.473340 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:12.474931 (+ 1591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":53,"threads_started":1}
W20260806 13:07:12.475070 27627 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:12.470533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:40691 (local address 127.13.224.126:33601)
0806 13:07:12.470612 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:12.470614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:12.470664 (+ 50us) server_negotiation.cc:407] Connection header received
0806 13:07:12.470726 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:12.470727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:12.470744 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:12.470797 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:12.471175 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:12.471937 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:12.472667 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:12.472860 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:12.472910 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:12.475016 (+ 2106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:40691: BlockingRecv error: recv got EOF from 127.13.224.67:40691 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 13:07:13 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:13 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:13.242740 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:13.238216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:58371)
0806 13:07:13.238344 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:13.238354 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:13.238403 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:13.238533 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:13.238534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:13.238696 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:13.238747 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:13.238748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:13.239574 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:13.239575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:13.240101 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:13.240103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:13.240461 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:13.240462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:13.240516 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:13.240918 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:13.240922 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:13.242680 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
W20260806 13:07:13.242843 27634 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:13.238280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:58371 (local address 127.13.224.126:33601)
0806 13:07:13.238407 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:13.238409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:13.238442 (+ 33us) server_negotiation.cc:407] Connection header received
0806 13:07:13.238463 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:13.238464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:13.238474 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:13.238521 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:13.238838 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:13.239486 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:13.240178 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:13.240374 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:13.240428 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:13.242769 (+ 2341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:58371: BlockingRecv error: recv got EOF from 127.13.224.65:58371 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":57,"threads_started":1}
Aug 06 13:07:13 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:13 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:13.361033 27636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:13.356859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:59537)
0806 13:07:13.356980 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:13.356990 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:13.357044 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:13.357151 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:13.357152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:13.357320 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:13.357369 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:13.357370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:13.357997 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:13.357998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:13.358633 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:13.358636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:13.358948 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:13.358948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:13.358985 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:13.359374 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:13.359378 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:13.360975 (+ 1597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
W20260806 13:07:13.361129 27634 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:13.356922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:59537 (local address 127.13.224.126:33601)
0806 13:07:13.356988 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:13.356990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:13.357031 (+ 41us) server_negotiation.cc:407] Connection header received
0806 13:07:13.357086 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:13.357087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:13.357097 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:13.357145 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:13.357422 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:13.357929 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:13.358699 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:13.358892 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:13.358946 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:13.361069 (+ 2123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:59537: BlockingRecv error: recv got EOF from 127.13.224.66:59537 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 13:07:13 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:13 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:13.479871 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:13.475433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:46975)
0806 13:07:13.475605 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:13.475617 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:13.475687 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:13.475829 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:13.475830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:13.476008 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:13.476065 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:13.476067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:13.476726 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:13.476727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:13.477253 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:13.477255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:13.477594 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:13.477596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:13.477656 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:13.478143 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:13.478147 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:13.479788 (+ 1641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
W20260806 13:07:13.479949 27634 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:13.475547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:46975 (local address 127.13.224.126:33601)
0806 13:07:13.475619 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:13.475621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:13.475681 (+ 60us) server_negotiation.cc:407] Connection header received
0806 13:07:13.475744 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:13.475744 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:13.475760 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:13.475815 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:13.476146 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:13.476643 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:13.477336 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:13.477524 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:13.477587 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:13.479895 (+ 2308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:46975: BlockingRecv error: recv got EOF from 127.13.224.67:46975 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 13:07:14 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:14 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:14.247721 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:14.243218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:47901)
0806 13:07:14.243405 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:14.243419 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:14.243479 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:14.243645 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:14.243646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:14.243805 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:14.243857 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:14.243858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:14.244495 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:14.244496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:14.245031 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:14.245032 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:14.245409 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:14.245409 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:14.245447 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:14.245879 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:14.245884 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:14.247632 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":89,"threads_started":1}
W20260806 13:07:14.247895 27639 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:14.243318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:47901 (local address 127.13.224.126:33601)
0806 13:07:14.243514 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:14.243516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:14.243526 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:07:14.243548 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:14.243549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:14.243560 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:14.243625 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:14.243916 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:14.244424 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:14.245120 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:14.245344 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:14.245408 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:14.247733 (+ 2325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:47901: BlockingRecv error: recv got EOF from 127.13.224.65:47901 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":74,"threads_started":1}
Aug 06 13:07:14 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:14 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:14.365908 27640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:14.361491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:57429)
0806 13:07:14.361660 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:14.361671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:14.361737 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:14.361850 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:14.361850 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:14.362054 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:14.362111 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:14.362113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:14.362709 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:14.362710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:14.363246 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:14.363248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:14.363529 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:14.363530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:14.363574 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:14.363974 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:14.363978 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:14.365846 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":88,"threads_started":1}
W20260806 13:07:14.365994 27639 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:14.361570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:57429 (local address 127.13.224.126:33601)
0806 13:07:14.361630 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:14.361632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:14.361719 (+ 87us) server_negotiation.cc:407] Connection header received
0806 13:07:14.361786 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:14.361786 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:14.361797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:14.361841 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:14.362159 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:14.362653 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:14.363307 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:14.363479 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:14.363522 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:14.365936 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:57429: BlockingRecv error: recv got EOF from 127.13.224.66:57429 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 13:07:14 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:14 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:14.484601 27641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:14.480337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:34329)
0806 13:07:14.480478 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:14.480490 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:14.480558 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:14.480683 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:14.480684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:14.480872 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:14.480918 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:14.480920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:14.481578 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:14.481580 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:14.482138 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:14.482140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:14.482471 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:14.482472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:14.482527 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:14.482921 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:14.482925 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:14.484522 (+ 1597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
W20260806 13:07:14.484686 27639 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:14.480406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:34329 (local address 127.13.224.126:33601)
0806 13:07:14.480468 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:14.480471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:14.480537 (+ 66us) server_negotiation.cc:407] Connection header received
0806 13:07:14.480608 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:14.480608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:14.480622 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:14.480669 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:14.480994 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:14.481501 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:14.482223 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:14.482410 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:14.482466 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:14.484627 (+ 2161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:34329: BlockingRecv error: recv got EOF from 127.13.224.67:34329 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 13:07:15 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:15 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:15.252683 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:15.248181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:40901)
0806 13:07:15.248327 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:15.248337 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:15.248389 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:15.248506 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:15.248507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:15.248669 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:15.248718 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:15.248720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:15.249344 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:15.249345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:15.249872 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:15.249874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:15.250223 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:15.250224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:15.250278 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:15.250714 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:15.250719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:15.252616 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
W20260806 13:07:15.252790 27644 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:15.248260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:40901 (local address 127.13.224.126:33601)
0806 13:07:15.248367 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:15.248368 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:15.248377 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:15.248430 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:15.248430 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:15.248441 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:15.248495 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:15.248768 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:15.249265 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:15.249933 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:15.250144 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:15.250187 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:15.252719 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:40901: BlockingRecv error: recv got EOF from 127.13.224.65:40901 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":60,"threads_started":1}
Aug 06 13:07:15 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:15 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:15.374279 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:15.366406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:39925)
0806 13:07:15.366539 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:15.366550 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:15.366606 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:15.366731 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:15.366731 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:15.366888 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:15.366938 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:15.366941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:15.367528 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:15.367528 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:15.368051 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:15.368052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:15.368358 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:15.368359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:15.368413 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:15.372526 (+ 4113us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:15.372531 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:15.374216 (+ 1685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
W20260806 13:07:15.374366 27644 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:15.366481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:39925 (local address 127.13.224.126:33601)
0806 13:07:15.366551 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:15.366553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:15.366601 (+ 48us) server_negotiation.cc:407] Connection header received
0806 13:07:15.366660 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:15.366661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:15.366673 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:15.366711 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:15.366989 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:15.367474 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:15.368104 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:15.368279 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:15.368321 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:15.374308 (+ 5987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:39925: BlockingRecv error: recv got EOF from 127.13.224.66:39925 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 13:07:15 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:15 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:15.489395 27646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:15.485067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:54909)
0806 13:07:15.485202 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:15.485214 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:15.485281 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:15.485412 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:15.485413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:15.485606 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:15.485655 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:15.485657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:15.486363 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:15.486364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:15.486894 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:15.486895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:15.487216 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:15.487217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:15.487274 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:15.487667 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:15.487670 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:15.489319 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
W20260806 13:07:15.489464 27644 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:15.485145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:54909 (local address 127.13.224.126:33601)
0806 13:07:15.485206 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:15.485209 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:15.485268 (+ 59us) server_negotiation.cc:407] Connection header received
0806 13:07:15.485336 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:15.485337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:15.485350 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:15.485395 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:15.485730 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:15.486286 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:15.486961 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:15.487156 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:15.487211 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:15.489410 (+ 2199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:54909: BlockingRecv error: recv got EOF from 127.13.224.67:54909 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 13:07:16 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:16 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:16.257688 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:16.253159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:40897)
0806 13:07:16.253309 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:16.253319 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:16.253368 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:16.253512 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:16.253512 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:16.253667 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:16.253725 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:16.253727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:16.254369 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:16.254371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:16.255041 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:16.255043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:16.255441 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:16.255443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:16.255488 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:16.255887 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:16.255891 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:16.257627 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":71,"threads_started":1}
W20260806 13:07:16.257795 27653 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:16.253231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:40897 (local address 127.13.224.126:33601)
0806 13:07:16.253349 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:16.253351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:16.253359 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:07:16.253429 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:16.253430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:16.253441 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:16.253501 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:16.253776 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:16.254287 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:16.255122 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:16.255374 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:16.255421 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:16.257722 (+ 2301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:40897: BlockingRecv error: recv got EOF from 127.13.224.65:40897 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
Aug 06 13:07:16 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:16 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:16.379200 27654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:16.374747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:57241)
0806 13:07:16.374930 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:16.374942 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:16.375008 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:16.375158 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:16.375159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:16.375329 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:16.375378 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:16.375380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:16.376023 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:16.376025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:16.376543 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:16.376545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:16.376872 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:16.376873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:16.376930 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:16.377354 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:16.377359 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:16.379110 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":90,"threads_started":1}
W20260806 13:07:16.379266 27653 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:16.374878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:57241 (local address 127.13.224.126:33601)
0806 13:07:16.374943 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:16.374945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:16.374999 (+ 54us) server_negotiation.cc:407] Connection header received
0806 13:07:16.375063 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:16.375064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:16.375075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:16.375144 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:16.375452 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:16.375952 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:16.376622 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:16.376814 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:16.376866 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:16.379214 (+ 2348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:57241: BlockingRecv error: recv got EOF from 127.13.224.66:57241 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 13:07:16 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:16 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:16.493976 27655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:16.489914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:37903)
0806 13:07:16.490081 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:16.490092 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:16.490157 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:16.490276 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:16.490277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:16.490439 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:16.490485 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:16.490487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:16.491066 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:16.491066 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:16.491595 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:16.491596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:16.491865 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:16.491866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:16.491903 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:16.492327 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:16.492332 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:16.493919 (+ 1587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
W20260806 13:07:16.494102 27653 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:16.489979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:37903 (local address 127.13.224.126:33601)
0806 13:07:16.490043 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:16.490045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:16.490152 (+ 107us) server_negotiation.cc:407] Connection header received
0806 13:07:16.490207 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:16.490207 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:16.490220 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:16.490257 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:16.490534 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:16.491015 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:16.491650 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:16.491819 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:16.491861 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:16.494041 (+ 2180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:37903: BlockingRecv error: recv got EOF from 127.13.224.67:37903 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 13:07:17 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:17 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:17.262526 27656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:17.258127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:34067)
0806 13:07:17.258251 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:17.258262 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:17.258308 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:17.258446 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:17.258447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:17.258600 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:17.258649 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:17.258651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:17.259283 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:17.259283 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:17.259805 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:17.259807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:17.260106 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:17.260107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:17.260141 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:17.260614 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:17.260620 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:17.262445 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
W20260806 13:07:17.262601 27657 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:17.258189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:34067 (local address 127.13.224.126:33601)
0806 13:07:17.258335 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:17.258338 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:17.258346 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:07:17.258369 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:17.258370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:17.258379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:17.258436 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:17.258699 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:17.259191 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:17.259865 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:17.260058 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:17.260104 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:17.262543 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:34067: BlockingRecv error: recv got EOF from 127.13.224.65:34067 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
Aug 06 13:07:17 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:17 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:17.384183 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:17.379668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:48395)
0806 13:07:17.379791 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:17.379802 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:17.379857 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:17.379974 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:17.379975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:17.380126 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:17.380174 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:17.380176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:17.380795 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:17.380795 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:17.381343 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:17.381345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:17.381714 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:17.381714 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:17.381751 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:17.382240 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:17.382245 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:17.384097 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
W20260806 13:07:17.384272 27657 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:17.379760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:48395 (local address 127.13.224.126:33601)
0806 13:07:17.379816 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:17.379818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:17.379846 (+ 28us) server_negotiation.cc:407] Connection header received
0806 13:07:17.379899 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:17.379900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:17.379911 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:17.379958 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:17.380238 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:17.380735 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:17.381449 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:17.381650 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:17.381711 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:17.384215 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:48395: BlockingRecv error: recv got EOF from 127.13.224.66:48395 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 13:07:17 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:17 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:17.498857 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:17.494433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:49771)
0806 13:07:17.494593 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:17.494605 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:17.494672 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:17.494816 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:17.494816 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:17.494979 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:17.495027 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:17.495029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:17.495618 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:17.495618 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:17.496134 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:17.496135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:17.496440 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:17.496441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:17.496478 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:17.496904 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:17.496909 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:17.498776 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
W20260806 13:07:17.498921 27657 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:17.494472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:49771 (local address 127.13.224.126:33601)
0806 13:07:17.494524 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:17.494526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:17.494671 (+ 145us) server_negotiation.cc:407] Connection header received
0806 13:07:17.494735 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:17.494735 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:17.494750 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:17.494806 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:17.495077 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:17.495565 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:17.496185 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:17.496390 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:17.496436 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:17.498870 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:49771: BlockingRecv error: recv got EOF from 127.13.224.67:49771 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 13:07:18 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:18 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:18.267100 27661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:18.262951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:53889)
0806 13:07:18.263073 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:18.263083 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:18.263130 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:18.263278 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:18.263279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:18.263436 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:18.263488 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:18.263489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:18.264078 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:18.264078 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:18.264609 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:18.264610 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:18.264897 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:18.264898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:18.264936 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:18.265332 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:18.265337 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:18.267039 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
W20260806 13:07:18.267275 27664 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:18.263016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:53889 (local address 127.13.224.126:33601)
0806 13:07:18.263164 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:18.263165 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:18.263173 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:07:18.263194 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:18.263195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:18.263205 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:18.263262 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:18.263539 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:18.264026 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:18.264667 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:18.264849 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:18.264891 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:18.267217 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:53889: BlockingRecv error: recv got EOF from 127.13.224.65:53889 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":57,"threads_started":1}
Aug 06 13:07:18 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:18 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:18.388998 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:18.384613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:37539)
0806 13:07:18.384732 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:18.384743 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:18.384800 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:18.384934 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:18.384934 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:18.385105 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:18.385161 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:18.385162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:18.385902 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:18.385903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:18.386489 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:18.386492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:18.386805 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:18.386805 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:18.386843 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:18.387236 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:18.387240 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:18.388927 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
W20260806 13:07:18.389123 27664 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:18.384702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:37539 (local address 127.13.224.126:33601)
0806 13:07:18.384760 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:18.384761 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:18.384788 (+ 27us) server_negotiation.cc:407] Connection header received
0806 13:07:18.384855 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:18.384855 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:18.384865 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:18.384922 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:18.385218 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:18.385824 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:18.386556 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:18.386750 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:18.386802 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:18.389049 (+ 2247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:37539: BlockingRecv error: recv got EOF from 127.13.224.66:37539 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 06 13:07:18 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:18 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:18.504216 27666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:18.499459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:50391)
0806 13:07:18.499622 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:18.499633 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:18.499690 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:18.499819 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:18.499820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:18.499994 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:18.500041 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:18.500042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:18.500671 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:18.500673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:18.501210 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:18.501212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:18.501536 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:18.501537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:18.501595 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:18.502132 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:18.502138 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:18.504120 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
W20260806 13:07:18.504267 27664 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:18.499536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:50391 (local address 127.13.224.126:33601)
0806 13:07:18.499582 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:18.499585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:18.499689 (+ 104us) server_negotiation.cc:407] Connection header received
0806 13:07:18.499743 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:18.499743 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:18.499757 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:18.499806 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:18.500109 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:18.500599 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:18.501287 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:18.501471 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:18.501525 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:18.504214 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:50391: BlockingRecv error: recv got EOF from 127.13.224.67:50391 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 06 13:07:19 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:19 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:19.271628 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:19.267547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:51955)
0806 13:07:19.267701 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:19.267711 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:19.267758 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:19.267900 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:19.267901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:19.268056 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:19.268105 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:19.268107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:19.268710 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:19.268710 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:19.269235 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:19.269237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:19.269511 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:19.269512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:19.269547 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:19.269929 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:19.269933 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:19.271570 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
W20260806 13:07:19.271727 27668 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:19.267657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:51955 (local address 127.13.224.126:33601)
0806 13:07:19.267790 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:19.267791 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:19.267799 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:07:19.267819 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:19.267820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:19.267829 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:19.267885 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:19.268156 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:19.268655 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:19.269290 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:19.269466 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:19.269508 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:19.271660 (+ 2152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:51955: BlockingRecv error: recv got EOF from 127.13.224.65:51955 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
Aug 06 13:07:19 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:19 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:19.393632 27669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:19.389433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:60081)
0806 13:07:19.389565 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:19.389575 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:19.389631 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:19.389747 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:19.389747 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:19.389910 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:19.389961 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:19.389962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:19.390732 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:19.390733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:19.391254 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:19.391255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:19.391542 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:19.391542 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:19.391579 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:19.391944 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:19.391948 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:19.393557 (+ 1609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
W20260806 13:07:19.393695 27668 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:19.389481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:60081 (local address 127.13.224.126:33601)
0806 13:07:19.389530 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:19.389532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:19.389623 (+ 91us) server_negotiation.cc:407] Connection header received
0806 13:07:19.389674 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:19.389675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:19.389686 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:19.389736 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:19.390008 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:19.390675 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:19.391321 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:19.391495 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:19.391537 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:19.393646 (+ 2109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:60081: BlockingRecv error: recv got EOF from 127.13.224.66:60081 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 06 13:07:19 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:19 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:19.508805 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:19.504656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:49581)
0806 13:07:19.504792 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:19.504802 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:19.504859 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:19.504967 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:19.504968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:19.505131 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:19.505180 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:19.505181 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:19.505799 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:19.505800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:19.506363 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:19.506365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:19.506640 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:19.506641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:19.506679 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:19.507064 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:19.507068 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:19.508726 (+ 1658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":59,"threads_started":1}
W20260806 13:07:19.508958 27668 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:19.504710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:49581 (local address 127.13.224.126:33601)
0806 13:07:19.504779 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:19.504781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:19.504855 (+ 74us) server_negotiation.cc:407] Connection header received
0806 13:07:19.504897 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:19.504897 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:19.504910 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:19.504954 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:19.505234 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:19.505742 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:19.506419 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:19.506594 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:19.506636 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:19.508854 (+ 2218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:49581: BlockingRecv error: recv got EOF from 127.13.224.67:49581 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 13:07:20 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:20 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:20.276439 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:20.272036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:56933)
0806 13:07:20.272160 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:20.272170 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:20.272217 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:20.272342 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:20.272343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:20.272502 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:20.272552 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:20.272554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:20.273165 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:20.273166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:20.273703 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:20.273704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:20.273991 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:20.273992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:20.274050 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:20.274436 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:20.274440 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:20.276380 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
W20260806 13:07:20.276533 27672 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:20.272104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:56933 (local address 127.13.224.126:33601)
0806 13:07:20.272235 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:20.272237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:20.272244 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:20.272263 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:20.272264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:20.272274 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:20.272333 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:20.272605 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:20.273099 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:20.273760 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:20.273944 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:20.273990 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:20.276466 (+ 2476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:56933: BlockingRecv error: recv got EOF from 127.13.224.65:56933 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
Aug 06 13:07:20 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:20 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:20.402875 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:20.394096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:56205)
0806 13:07:20.394219 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:20.394229 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:20.394286 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:20.394555 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:20.394555 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:20.394710 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:20.394763 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:20.394765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:20.395407 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:20.395407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:20.395917 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:20.395918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:20.396234 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:20.396234 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:20.396278 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:20.400937 (+ 4659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:20.400942 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:20.402816 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
W20260806 13:07:20.402961 27672 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:20.394322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:56205 (local address 127.13.224.126:33601)
0806 13:07:20.394440 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:20.394442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:20.394450 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:07:20.394473 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:20.394474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:20.394485 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:20.394533 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:20.394838 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:20.395348 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:20.395979 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:20.396180 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:20.396230 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:20.402897 (+ 6667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:56205: BlockingRecv error: recv got EOF from 127.13.224.66:56205 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 06 13:07:20 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:20 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:20.513355 27675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:20.509210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:47349)
0806 13:07:20.509376 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:20.509387 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:20.509449 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:20.509571 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:20.509571 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:20.509730 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:20.509779 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:20.509782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:20.510382 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:20.510383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:20.510887 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:20.510889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:20.511152 (+ 263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:20.511153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:20.511189 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:20.511574 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:20.511578 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:20.513297 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
W20260806 13:07:20.513448 27672 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:20.509249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:47349 (local address 127.13.224.126:33601)
0806 13:07:20.509302 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:20.509305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:20.509463 (+ 158us) server_negotiation.cc:407] Connection header received
0806 13:07:20.509490 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:20.509491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:20.509505 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:20.509561 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:20.509830 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:20.510333 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:20.510940 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:20.511107 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:20.511147 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:20.513389 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:47349: BlockingRecv error: recv got EOF from 127.13.224.67:47349 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 13:07:21 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:21 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:21.281168 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:21.276887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:44533)
0806 13:07:21.277014 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:21.277027 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:21.277085 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:21.277216 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:21.277217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:21.277384 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:21.277432 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:21.277434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:21.278055 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:21.278056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:21.278571 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:21.278572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:21.278951 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:21.278952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:21.278987 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:21.279491 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:21.279495 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:21.281097 (+ 1602us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
W20260806 13:07:21.281276 27677 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:21.276950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:44533 (local address 127.13.224.126:33601)
0806 13:07:21.277089 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:21.277090 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:21.277099 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:21.277144 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:21.277144 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:21.277156 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:21.277203 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:21.277484 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:21.277973 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:21.278663 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:21.278885 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:21.278955 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:21.281207 (+ 2252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:44533: BlockingRecv error: recv got EOF from 127.13.224.65:44533 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
Aug 06 13:07:21 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:21 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:21.407804 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:21.403413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:39517)
0806 13:07:21.403570 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:21.403583 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:21.403657 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:21.403791 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:21.403792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:21.403964 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:21.404013 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:21.404014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:21.404662 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:21.404663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:21.405196 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:21.405198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:21.405538 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:21.405540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:21.405595 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:21.405989 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:21.405993 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:21.407736 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
W20260806 13:07:21.407904 27677 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:21.403520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:39517 (local address 127.13.224.126:33601)
0806 13:07:21.403611 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:21.403613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:21.403639 (+ 26us) server_negotiation.cc:407] Connection header received
0806 13:07:21.403712 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:21.403713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:21.403727 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:21.403773 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:21.404088 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:21.404578 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:21.405277 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:21.405477 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:21.405534 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:21.407833 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:39517: BlockingRecv error: recv got EOF from 127.13.224.66:39517 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 13:07:21 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:21 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:21.518436 27681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:21.513842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:46375)
0806 13:07:21.514003 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:21.514036 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:21.514087 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:21.514217 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:21.514218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:21.514386 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:21.514433 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:21.514435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:21.515201 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:21.515202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:21.515758 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:21.515761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:21.516131 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:21.516132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:21.516192 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:21.516663 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:21.516668 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:21.518349 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":64,"threads_started":1}
W20260806 13:07:21.518507 27677 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:21.513953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:46375 (local address 127.13.224.126:33601)
0806 13:07:21.514055 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:21.514057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:21.514071 (+ 14us) server_negotiation.cc:407] Connection header received
0806 13:07:21.514138 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:21.514139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:21.514153 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:21.514207 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:21.514489 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:21.515119 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:21.515850 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:21.516059 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:21.516123 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:21.518450 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:46375: BlockingRecv error: recv got EOF from 127.13.224.67:46375 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 13:07:22 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:22 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:22.285797 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:22.281616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:58525)
0806 13:07:22.281754 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:22.281765 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:22.281816 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:22.281945 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:22.281945 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:22.282122 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:22.282169 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:22.282170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:22.282783 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:22.282783 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:22.283304 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:22.283305 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:22.283597 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:22.283598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:22.283633 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:22.284038 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:22.284041 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:22.285720 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
W20260806 13:07:22.285873 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:22.281715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:58525 (local address 127.13.224.126:33601)
0806 13:07:22.281830 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:22.281832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:22.281841 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:22.281865 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:22.281866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:22.281876 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:22.281934 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:22.282221 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:22.282730 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:22.283356 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:22.283550 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:22.283594 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:22.285812 (+ 2218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:58525: BlockingRecv error: recv got EOF from 127.13.224.65:58525 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
Aug 06 13:07:22 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:22 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:22.412616 27685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:22.408279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:33859)
0806 13:07:22.408410 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:22.408420 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:22.408481 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:22.408608 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:22.408609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:22.408790 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:22.408844 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:22.408846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:22.409461 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:22.409462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:22.410009 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:22.410011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:22.410350 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:22.410350 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:22.410392 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:22.410808 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:22.410812 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:22.412536 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
W20260806 13:07:22.412699 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:22.408324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:33859 (local address 127.13.224.126:33601)
0806 13:07:22.408374 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:22.408376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:22.408470 (+ 94us) server_negotiation.cc:407] Connection header received
0806 13:07:22.408533 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:22.408534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:22.408546 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:22.408592 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:22.408898 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:22.409396 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:22.410109 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:22.410295 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:22.410343 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:22.412637 (+ 2294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:33859: BlockingRecv error: recv got EOF from 127.13.224.66:33859 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 06 13:07:22 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:22 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:22.523362 27687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:22.518905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:60029)
0806 13:07:22.519035 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:22.519047 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:22.519104 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:22.519222 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:22.519223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:22.519398 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:22.519443 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:22.519445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:22.520034 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:22.520034 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:22.520623 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:22.520627 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:22.521011 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:22.521012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:22.521060 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:22.521484 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:22.521489 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:22.523293 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
W20260806 13:07:22.523443 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:22.518979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:60029 (local address 127.13.224.126:33601)
0806 13:07:22.519051 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:22.519052 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:22.519100 (+ 48us) server_negotiation.cc:407] Connection header received
0806 13:07:22.519158 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:22.519158 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:22.519171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:22.519211 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:22.519490 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:22.519984 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:22.520695 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:22.520945 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:22.521011 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:22.523384 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:60029: BlockingRecv error: recv got EOF from 127.13.224.67:60029 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 13:07:23 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:23 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:23.290617 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:23.286249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:52477)
0806 13:07:23.286377 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:23.286387 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:23.286438 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:23.286557 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:23.286558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:23.286725 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:23.286783 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:23.286785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:23.287446 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:23.287446 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:23.288014 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:23.288017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:23.288372 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:23.288373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:23.288414 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:23.288852 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:23.288856 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:23.290550 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
W20260806 13:07:23.290723 27690 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:23.286343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:52477 (local address 127.13.224.126:33601)
0806 13:07:23.286449 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:23.286450 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:23.286471 (+ 21us) server_negotiation.cc:407] Connection header received
0806 13:07:23.286491 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:23.286491 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:23.286501 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:23.286548 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:23.286845 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:23.287359 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:23.288093 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:23.288317 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:23.288366 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:23.290647 (+ 2281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:52477: BlockingRecv error: recv got EOF from 127.13.224.65:52477 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":52,"threads_started":1}
Aug 06 13:07:23 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:23 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:23.417280 27691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:23.413068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:35111)
0806 13:07:23.413220 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:23.413230 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:23.413274 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:23.413401 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:23.413402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:23.413568 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:23.413617 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:23.413618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:23.414304 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:23.414305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:23.414851 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:23.414852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:23.415157 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:23.415158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:23.415194 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:23.415601 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:23.415605 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:23.417222 (+ 1617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
W20260806 13:07:23.417362 27690 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:23.413194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:35111 (local address 127.13.224.126:33601)
0806 13:07:23.413254 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:23.413256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:23.413265 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:07:23.413322 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:23.413323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:23.413333 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:23.413390 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:23.413679 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:23.414232 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:23.414912 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:23.415105 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:23.415155 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:23.417309 (+ 2154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:35111: BlockingRecv error: recv got EOF from 127.13.224.66:35111 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 13:07:23 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:23 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:23.527946 27692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:23.523801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:51675)
0806 13:07:23.523951 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:23.523961 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:23.524016 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:23.524139 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:23.524140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:23.524321 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:23.524366 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:23.524368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:23.525082 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:23.525083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:23.525620 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:23.525622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:23.525911 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:23.525912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:23.525948 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:23.526339 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:23.526343 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:23.527887 (+ 1544us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
W20260806 13:07:23.528028 27690 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:23.523891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:51675 (local address 127.13.224.126:33601)
0806 13:07:23.523968 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:23.523970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:23.524012 (+ 42us) server_negotiation.cc:407] Connection header received
0806 13:07:23.524055 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:23.524055 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:23.524067 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:23.524123 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:23.524422 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:23.525009 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:23.525676 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:23.525862 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:23.525909 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:23.527968 (+ 2059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:51675: BlockingRecv error: recv got EOF from 127.13.224.67:51675 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 13:07:24 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:24 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:24.296690 27693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:24.291073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:44975)
0806 13:07:24.291205 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:24.291216 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:24.291277 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:24.291430 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:24.291431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:24.291637 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:24.291694 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:24.291696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:24.292629 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:24.292630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:24.293437 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:24.293441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:24.293964 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:24.293965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:24.294063 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:24.294541 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:24.294547 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:24.296589 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
W20260806 13:07:24.296803 27695 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:24.291104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:44975 (local address 127.13.224.126:33601)
0806 13:07:24.291224 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:24.291226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:24.291277 (+ 51us) server_negotiation.cc:407] Connection header received
0806 13:07:24.291340 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:24.291340 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:24.291354 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:24.291424 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:24.291822 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:24.292540 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:24.293549 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:24.293881 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:24.293955 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:24.296698 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:44975: BlockingRecv error: recv got EOF from 127.13.224.65:44975 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
Aug 06 13:07:24 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:24 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:24.421963 27696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:24.417744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:60389)
0806 13:07:24.417884 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:24.417896 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:24.417958 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:24.418084 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:24.418085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:24.418260 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:24.418312 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:24.418314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:24.418893 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:24.418894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:24.419447 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:24.419449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:24.419741 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:24.419742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:24.419781 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:24.420173 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:24.420179 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:24.421834 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
W20260806 13:07:24.421999 27695 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:24.417830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:60389 (local address 127.13.224.126:33601)
0806 13:07:24.417876 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:24.417878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:24.417940 (+ 62us) server_negotiation.cc:407] Connection header received
0806 13:07:24.418004 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:24.418004 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:24.418029 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:24.418072 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:24.418366 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:24.418840 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:24.419512 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:24.419691 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:24.419730 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:24.421952 (+ 2222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:60389: BlockingRecv error: recv got EOF from 127.13.224.66:60389 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 06 13:07:24 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:24 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:24.532495 27697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:24.528397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:53841)
0806 13:07:24.528518 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:24.528528 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:24.528585 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:24.528713 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:24.528713 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:24.528875 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:24.528921 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:24.528922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:24.529509 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:24.529509 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:24.530058 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:24.530060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:24.530397 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:24.530397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:24.530433 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:24.530812 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:24.530816 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:24.532427 (+ 1611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
W20260806 13:07:24.532568 27695 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:24.528453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:53841 (local address 127.13.224.126:33601)
0806 13:07:24.528531 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:24.528533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:24.528570 (+ 37us) server_negotiation.cc:407] Connection header received
0806 13:07:24.528637 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:24.528638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:24.528650 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:24.528697 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:24.528968 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:24.529455 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:24.530141 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:24.530337 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:24.530396 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:24.532512 (+ 2116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:53841: BlockingRecv error: recv got EOF from 127.13.224.67:53841 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 13:07:25 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:25 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:25.301530 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:25.297215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:56453)
0806 13:07:25.297351 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:25.297429 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:25.297494 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:25.297638 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:25.297638 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:25.297809 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:25.297858 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:25.297860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:25.298579 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:25.298580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:25.299099 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:25.299100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:25.299394 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:25.299395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:25.299430 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:25.299879 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:25.299884 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:25.301470 (+ 1586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
W20260806 13:07:25.301626 27700 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:25.297278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:56453 (local address 127.13.224.126:33601)
0806 13:07:25.297409 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:25.297410 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:25.297474 (+ 64us) server_negotiation.cc:407] Connection header received
0806 13:07:25.297547 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:25.297548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:25.297561 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:25.297623 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:25.297909 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:25.298514 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:25.299158 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:25.299348 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:25.299390 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:25.301556 (+ 2166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:56453: BlockingRecv error: recv got EOF from 127.13.224.65:56453 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
Aug 06 13:07:25 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:25 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:25.429970 27703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:25.422367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:45405)
0806 13:07:25.422490 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:25.422500 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:25.422553 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:25.422676 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:25.422677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:25.422845 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:25.422895 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:25.422896 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:25.423538 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:25.423539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:25.424058 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:25.424060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:25.424467 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:25.424469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:25.424534 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:25.428191 (+ 3657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:25.428197 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:25.429908 (+ 1711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
W20260806 13:07:25.430182 27700 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:25.422424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:45405 (local address 127.13.224.126:33601)
0806 13:07:25.422479 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:25.422481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:25.422538 (+ 57us) server_negotiation.cc:407] Connection header received
0806 13:07:25.422607 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:25.422607 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:25.422624 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:25.422665 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:25.422946 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:25.423476 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:25.424125 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:25.424383 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:25.424443 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:25.430088 (+ 5645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:45405: BlockingRecv error: recv got EOF from 127.13.224.66:45405 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 13:07:25 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:25 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.67@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:25.537271 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:25.532979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.67:48105)
0806 13:07:25.533139 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:25.533151 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:25.533216 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:25.533345 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:25.533345 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:25.533523 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:25.533583 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:25.533586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:25.534249 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:25.534250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:25.534784 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:25.534786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:25.535086 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:25.535086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:25.535123 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:25.535536 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:25.535540 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:25.537210 (+ 1670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":106,"threads_started":1}
W20260806 13:07:25.537370 27700 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:25.533026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:48105 (local address 127.13.224.126:33601)
0806 13:07:25.533075 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:25.533077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:25.533205 (+ 128us) server_negotiation.cc:407] Connection header received
0806 13:07:25.533265 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:25.533265 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:25.533280 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:25.533330 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:25.533636 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:25.534169 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:25.534859 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:25.535037 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:25.535083 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:25.537299 (+ 2216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.67:48105: BlockingRecv error: recv got EOF from 127.13.224.67:48105 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 06 13:07:26 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:26 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.65@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:26.306329 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:26.302010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.65:55703)
0806 13:07:26.302166 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:26.302177 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:26.302227 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:26.302346 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:26.302347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:26.302507 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:26.302555 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:26.302556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.303274 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.303276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:26.303887 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:26.303890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.304215 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.304216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:26.304266 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:26.304661 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:26.304665 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:26.306269 (+ 1604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
W20260806 13:07:26.306435 27711 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:26.302076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:55703 (local address 127.13.224.126:33601)
0806 13:07:26.302201 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:26.302203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:26.302214 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:07:26.302270 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:26.302270 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:26.302280 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:26.302335 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:26.302615 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.303180 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.303955 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.304159 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.304207 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:26.306363 (+ 2156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.65:55703: BlockingRecv error: recv got EOF from 127.13.224.65:55703 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
Aug 06 13:07:26 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 13:07:26 dist-test-slave-q9h9 krb5kdc[26211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.224.66@KRBTEST.COM for oryx/127.13.224.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 13:07:26.435070 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:26.430550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:33601 (local address 127.13.224.66:43043)
0806 13:07:26.430674 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:26.430686 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:26.430752 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:26.430879 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:26.430881 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:26.431064 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:26.431115 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:26.431117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.431778 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.431780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:26.432337 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:26.432339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.432680 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.432680 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:26.432746 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:26.433182 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:26.433187 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:26.434990 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.224.126:33601: Server oryx/127.13.224.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
W20260806 13:07:26.435148 27711 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:26.430589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:43043 (local address 127.13.224.126:33601)
0806 13:07:26.430645 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:26.430647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:26.430736 (+ 89us) server_negotiation.cc:407] Connection header received
0806 13:07:26.430805 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:26.430806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:26.430818 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:26.430866 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:26.431200 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.431706 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.432423 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.432613 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.432675 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:26.435096 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.224.66:43043: BlockingRecv error: recv got EOF from 127.13.224.66:43043 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:07:26.472455 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 27033
I20260806 13:07:26.484539 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 27245
I20260806 13:07:26.508608 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 27437
I20260806 13:07:26.514605 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 26929
2026-08-06T13:07:26Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17529 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T13:07:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:07:26Z Disabled control of system clock
I20260806 13:07:26.545342 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:37333
--webserver_interface=127.13.224.126
--webserver_port=0
--builtin_ntp_servers=127.13.224.84:46171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:37333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 13:07:26.624249 27718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:26.624429 27718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:26.624447 27718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:26.625597 27718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:07:26.625636 27718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:26.625648 27718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:26.625658 27718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:07:26.625669 27718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:07:26.627028 27718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:37333
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:37333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:26.627223 27718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:26.627451 27718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:26.632087 27734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:26.632197 27730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:26.632206 27718 server_base.cc:1034] running on GCE node
W20260806 13:07:26.632088 27732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:26.632536 27718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:26.632812 27718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:26.633952 27718 hybrid_clock.cc:648] HybridClock initialized: now 1786021646633935 us; error 35 us; skew 500 ppm
I20260806 13:07:26.635136 27718 webserver.cc:460] Webserver started at http://127.13.224.126:41335/ using document root <none> and password file <none>
I20260806 13:07:26.635357 27718 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:26.635406 27718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:26.635555 27718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:26.636579 27718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "a6e6bc2a4fbf43d3a504cae76cfb264a"
format_stamp: "Formatted at 2026-08-06 13:07:26 on dist-test-slave-q9h9"
server_key: "c95d13570134a91d63fe742e790a7c3a"
server_key_iv: "d7b09d6fa61c41ac6854fafc13b3c853"
server_key_version: "encryptionkey@0"
I20260806 13:07:26.636912 27718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "a6e6bc2a4fbf43d3a504cae76cfb264a"
format_stamp: "Formatted at 2026-08-06 13:07:26 on dist-test-slave-q9h9"
server_key: "c95d13570134a91d63fe742e790a7c3a"
server_key_iv: "d7b09d6fa61c41ac6854fafc13b3c853"
server_key_version: "encryptionkey@0"
I20260806 13:07:26.638269 27718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:07:26.638991 27744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:26.639122 27718 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:26.639205 27718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "a6e6bc2a4fbf43d3a504cae76cfb264a"
format_stamp: "Formatted at 2026-08-06 13:07:26 on dist-test-slave-q9h9"
server_key: "c95d13570134a91d63fe742e790a7c3a"
server_key_iv: "d7b09d6fa61c41ac6854fafc13b3c853"
server_key_version: "encryptionkey@0"
I20260806 13:07:26.639276 27718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:26.656198 27718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:26.656454 27718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:26.656598 27718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:26.660535 27718 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:37333
I20260806 13:07:26.660565 27819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:37333 every 8 connection(s)
I20260806 13:07:26.660940 27718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:07:26.661612 27820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:26.663975 27820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a: Bootstrap starting.
I20260806 13:07:26.664495 27820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:26.664732 27820 log.cc:824] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:26.665341 27820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a: No bootstrap required, opened a new log
I20260806 13:07:26.666814 27820 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e6bc2a4fbf43d3a504cae76cfb264a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37333 } }
I20260806 13:07:26.666944 27820 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:26.666965 27820 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6e6bc2a4fbf43d3a504cae76cfb264a, State: Initialized, Role: FOLLOWER
I20260806 13:07:26.667066 27820 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e6bc2a4fbf43d3a504cae76cfb264a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37333 } }
I20260806 13:07:26.667141 27820 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:07:26.667169 27820 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:07:26.667199 27820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:26.667654 27820 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e6bc2a4fbf43d3a504cae76cfb264a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37333 } }
I20260806 13:07:26.667734 27820 leader_election.cc:302] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6e6bc2a4fbf43d3a504cae76cfb264a; no voters:
I20260806 13:07:26.667850 27820 leader_election.cc:288] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:07:26.667918 27824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:07:26.668074 27824 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [term 1 LEADER]: Becoming Leader. State: Replica: a6e6bc2a4fbf43d3a504cae76cfb264a, State: Running, Role: LEADER
I20260806 13:07:26.668152 27820 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:07:26.668226 27824 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e6bc2a4fbf43d3a504cae76cfb264a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37333 } }
I20260806 13:07:26.668566 27828 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6e6bc2a4fbf43d3a504cae76cfb264a. Latest consensus state: current_term: 1 leader_uuid: "a6e6bc2a4fbf43d3a504cae76cfb264a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e6bc2a4fbf43d3a504cae76cfb264a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37333 } } }
I20260806 13:07:26.668691 27825 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6e6bc2a4fbf43d3a504cae76cfb264a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e6bc2a4fbf43d3a504cae76cfb264a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37333 } } }
I20260806 13:07:26.668712 27828 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:26.668782 27825 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:26.669227 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 27718
I20260806 13:07:26.669332 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:07:26.669528 14209 external_mini_cluster.cc:1428] Setting key e377397d2b1e833749d45e0453205610
W20260806 13:07:26.669891 27849 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:07:26.669942 27849 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:07:26.669986 27838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:07:26.670202 27838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:07:26.672235 27838 catalog_manager.cc:1347] Generated new cluster ID: cd9e20c3cc4347e9a210cb0230bf301e
I20260806 13:07:26.672281 27838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:07:26.675252 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:26.669932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40364 (local address 127.13.224.126:37333)
0806 13:07:26.670114 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:26.670123 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:26.670303 (+ 180us) server_negotiation.cc:407] Connection header received
0806 13:07:26.670688 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:26.670697 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:26.670771 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:26.670918 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:26.671123 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.672128 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.673227 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.673818 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.673932 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:26.674593 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:26.674597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:26.674597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:07:26.674783 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:26.674833 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:07:26.674846 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:26.674901 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:26.674980 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 13:07:26.675049 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":54,"threads_started":1}
I20260806 13:07:26.678591 27838 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:07:26.679049 27838 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:07:26.683137 27838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6e6bc2a4fbf43d3a504cae76cfb264a: Generated new TSK 0
I20260806 13:07:26.683269 27838 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:07:26.687767 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37333
--builtin_ntp_servers=127.13.224.84:46171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 13:07:26.768471 27854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:26.768656 27854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:26.768682 27854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:26.769860 27854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:26.769896 27854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:26.769920 27854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:07:26.771246 27854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:26.771447 27854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:26.771729 27854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:26.777336 27862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:26.777364 27864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:26.777756 27866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:26.778258 27854 server_base.cc:1034] running on GCE node
I20260806 13:07:26.778448 27854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:26.778673 27854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:26.779829 27854 hybrid_clock.cc:648] HybridClock initialized: now 1786021646779817 us; error 27 us; skew 500 ppm
I20260806 13:07:26.780956 27854 webserver.cc:460] Webserver started at http://127.13.224.65:43037/ using document root <none> and password file <none>
I20260806 13:07:26.781177 27854 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:26.781232 27854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:26.781356 27854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:26.782464 27854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "7fd7d31784b844068656b598831b4de4"
format_stamp: "Formatted at 2026-08-06 13:07:26 on dist-test-slave-q9h9"
server_key: "da6b2bc02dcb59362d9f641f4d77e0dd"
server_key_iv: "182427d4d17d97a942d20e84d38e2d6b"
server_key_version: "encryptionkey@0"
I20260806 13:07:26.782799 27854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "7fd7d31784b844068656b598831b4de4"
format_stamp: "Formatted at 2026-08-06 13:07:26 on dist-test-slave-q9h9"
server_key: "da6b2bc02dcb59362d9f641f4d77e0dd"
server_key_iv: "182427d4d17d97a942d20e84d38e2d6b"
server_key_version: "encryptionkey@0"
I20260806 13:07:26.784045 27854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:07:26.784723 27877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:26.784875 27854 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:26.784956 27854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "7fd7d31784b844068656b598831b4de4"
format_stamp: "Formatted at 2026-08-06 13:07:26 on dist-test-slave-q9h9"
server_key: "da6b2bc02dcb59362d9f641f4d77e0dd"
server_key_iv: "182427d4d17d97a942d20e84d38e2d6b"
server_key_version: "encryptionkey@0"
I20260806 13:07:26.785019 27854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:26.798310 27854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:26.798547 27854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:26.798641 27854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:26.798813 27854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:26.799075 27854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:26.799104 27854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:26.799129 27854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:26.799145 27854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:26.804517 27854 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:42997
I20260806 13:07:26.804582 28063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:42997 every 8 connection(s)
I20260806 13:07:26.804859 27854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:07:26.809397 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:26.805527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:35115 (local address 127.13.224.126:37333)
0806 13:07:26.805590 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:26.805592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:26.805790 (+ 198us) server_negotiation.cc:407] Connection header received
0806 13:07:26.806313 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:26.806315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:26.806325 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:26.806372 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:26.806951 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.807429 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.808301 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.808485 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.808530 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:26.809191 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:26.809193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:26.809193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:07:26.809198 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:26.809212 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:07:26.809217 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:26.809255 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:26.809316 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 13:07:26.809339 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 13:07:26.809577 28067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:26.805509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37333 (local address 127.13.224.65:35115)
0806 13:07:26.805685 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:26.805717 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:26.806226 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:26.806424 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:26.806427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:26.806435 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:26.806890 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:26.806895 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.807488 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.807488 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:26.808237 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:26.808239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.808544 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.808545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:26.808642 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:26.809071 (+ 429us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:07:26.809071 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:26.809098 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:07:26.809118 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:26.809260 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:26.809262 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:26.809265 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:07:26.809309 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 13:07:26.809385 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260806 13:07:26.810056 28064 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37333
I20260806 13:07:26.810139 28064 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:26.810300 28064 heartbeater.cc:511] Master 127.13.224.126:37333 requested a full tablet report, sending...
I20260806 13:07:26.810751 27767 ts_manager.cc:194] Registered new tserver with Master: 7fd7d31784b844068656b598831b4de4 (127.13.224.65:42997)
I20260806 13:07:26.811343 27767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.224.65:35115
I20260806 13:07:26.812860 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 27854
I20260806 13:07:26.812938 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:07:26.813113 14209 external_mini_cluster.cc:1428] Setting key f04101ea07e1731c07b54e35675dcaf7
I20260806 13:07:26.813853 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37333
--builtin_ntp_servers=127.13.224.84:46171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 13:07:26.893968 28068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:26.894188 28068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:26.894207 28068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:26.895340 28068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:26.895377 28068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:26.895404 28068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:07:26.896705 28068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:26.896941 28068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:26.897208 28068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:26.901957 28074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:26.901968 28068 server_base.cc:1034] running on GCE node
W20260806 13:07:26.901937 28078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:26.901917 28075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:26.902347 28068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:26.902565 28068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:26.903700 28068 hybrid_clock.cc:648] HybridClock initialized: now 1786021646903685 us; error 34 us; skew 500 ppm
I20260806 13:07:26.904892 28068 webserver.cc:460] Webserver started at http://127.13.224.66:35965/ using document root <none> and password file <none>
I20260806 13:07:26.905124 28068 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:26.905171 28068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:26.905300 28068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:26.906376 28068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "4a117ccfafd941a4a5185fc910a62ee0"
format_stamp: "Formatted at 2026-08-06 13:07:26 on dist-test-slave-q9h9"
server_key: "f8da2fc73c9680d361c9dd4b11142887"
server_key_iv: "c6d96733aed4a91fce3f59fc00785ddf"
server_key_version: "encryptionkey@0"
I20260806 13:07:26.906694 28068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "4a117ccfafd941a4a5185fc910a62ee0"
format_stamp: "Formatted at 2026-08-06 13:07:26 on dist-test-slave-q9h9"
server_key: "f8da2fc73c9680d361c9dd4b11142887"
server_key_iv: "c6d96733aed4a91fce3f59fc00785ddf"
server_key_version: "encryptionkey@0"
I20260806 13:07:26.907930 28068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:07:26.908577 28088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:26.908752 28068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:26.908833 28068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "4a117ccfafd941a4a5185fc910a62ee0"
format_stamp: "Formatted at 2026-08-06 13:07:26 on dist-test-slave-q9h9"
server_key: "f8da2fc73c9680d361c9dd4b11142887"
server_key_iv: "c6d96733aed4a91fce3f59fc00785ddf"
server_key_version: "encryptionkey@0"
I20260806 13:07:26.908895 28068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:26.932360 28068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:26.932638 28068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:26.932767 28068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:26.932971 28068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:26.933295 28068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:26.933329 28068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:26.933358 28068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:26.933382 28068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:26.938889 28068 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:40353
I20260806 13:07:26.938947 28249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:40353 every 8 connection(s)
I20260806 13:07:26.939239 28068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:07:26.944047 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:26.939877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:37345 (local address 127.13.224.126:37333)
0806 13:07:26.939935 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:26.939937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:26.940151 (+ 214us) server_negotiation.cc:407] Connection header received
0806 13:07:26.940636 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:26.940637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:26.940651 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:26.940698 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:26.941310 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.941791 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.942827 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.943010 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.943088 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:26.943847 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:26.943847 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:26.943848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:07:26.943852 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:26.943863 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:07:26.943867 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:26.943905 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:26.943965 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 13:07:26.943989 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 13:07:26.944234 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:26.939871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37333 (local address 127.13.224.66:37345)
0806 13:07:26.940034 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:26.940067 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:26.940546 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:26.940756 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:26.940759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:26.940767 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:26.941249 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:26.941254 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.941860 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.941861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:26.942742 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:26.942745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:26.943097 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:26.943098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:26.943211 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:26.943719 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:07:26.943719 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:26.943746 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:07:26.943774 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:26.943912 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:26.943914 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:26.943916 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:07:26.943958 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 13:07:26.944031 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260806 13:07:26.944702 28250 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37333
I20260806 13:07:26.944792 28250 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:26.944962 28250 heartbeater.cc:511] Master 127.13.224.126:37333 requested a full tablet report, sending...
I20260806 13:07:26.945317 27767 ts_manager.cc:194] Registered new tserver with Master: 4a117ccfafd941a4a5185fc910a62ee0 (127.13.224.66:40353)
I20260806 13:07:26.945641 27767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.224.66:37345
I20260806 13:07:26.948887 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 28068
I20260806 13:07:26.948968 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:07:26.949141 14209 external_mini_cluster.cc:1428] Setting key d2f005ed16bcaaf94be3f7613b3e02ad
I20260806 13:07:26.949919 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37333
--builtin_ntp_servers=127.13.224.84:46171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 13:07:27.031404 28253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:27.031594 28253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:27.031615 28253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:27.032747 28253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:27.032785 28253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:27.032809 28253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:07:27.034163 28253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:27.034389 28253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:27.034709 28253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:27.039631 28262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:27.039669 28260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:27.039829 28253 server_base.cc:1034] running on GCE node
W20260806 13:07:27.039891 28259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:27.040097 28253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:27.040325 28253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:27.041471 28253 hybrid_clock.cc:648] HybridClock initialized: now 1786021647041451 us; error 31 us; skew 500 ppm
I20260806 13:07:27.042840 28253 webserver.cc:460] Webserver started at http://127.13.224.67:42049/ using document root <none> and password file <none>
I20260806 13:07:27.043102 28253 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:27.043152 28253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:27.043285 28253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:27.044306 28253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "12279783158849d1b244d428c9b26f2e"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "38f2e6f2a18eda455cee8698a11ed90d"
server_key_iv: "9aca1c8025fd211f81d3ce8e3465edb5"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.044641 28253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "12279783158849d1b244d428c9b26f2e"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "38f2e6f2a18eda455cee8698a11ed90d"
server_key_iv: "9aca1c8025fd211f81d3ce8e3465edb5"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.045931 28253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:07:27.046643 28274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.046803 28253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:27.046909 28253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "12279783158849d1b244d428c9b26f2e"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "38f2e6f2a18eda455cee8698a11ed90d"
server_key_iv: "9aca1c8025fd211f81d3ce8e3465edb5"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.046972 28253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:27.061934 28253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:27.062263 28253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:27.062389 28253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:27.062585 28253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:27.062887 28253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:27.062922 28253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.062949 28253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:27.062968 28253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.068491 28253 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:44179
I20260806 13:07:27.068548 28434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:44179 every 8 connection(s)
I20260806 13:07:27.068851 28253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:07:27.073448 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:27.069531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:44063 (local address 127.13.224.126:37333)
0806 13:07:27.069584 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:27.069586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:27.069865 (+ 279us) server_negotiation.cc:407] Connection header received
0806 13:07:27.070338 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:27.070339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:27.070349 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:27.070399 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:27.070997 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:27.071469 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:27.072358 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:27.072526 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:27.072567 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:27.073273 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:27.073274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:27.073274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:07:27.073277 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:27.073287 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:07:27.073290 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:27.073317 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:27.073371 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 13:07:27.073389 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:07:27.073644 28438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:27.069533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37333 (local address 127.13.224.67:44063)
0806 13:07:27.069743 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:27.069776 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:27.070257 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:27.070451 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:27.070454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:27.070462 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:27.070941 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:27.070945 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:27.071526 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:27.071526 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:27.072302 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:27.072304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:27.072575 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:27.072576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:27.072675 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:27.073161 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:07:27.073161 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:27.073192 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:07:27.073214 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:27.073323 (+ 109us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:27.073325 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:27.073328 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:07:27.073369 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 13:07:27.073444 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260806 13:07:27.074103 28436 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37333
I20260806 13:07:27.074191 28436 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:27.074338 28436 heartbeater.cc:511] Master 127.13.224.126:37333 requested a full tablet report, sending...
I20260806 13:07:27.074682 27767 ts_manager.cc:194] Registered new tserver with Master: 12279783158849d1b244d428c9b26f2e (127.13.224.67:44179)
I20260806 13:07:27.075031 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 28253
I20260806 13:07:27.075122 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:07:27.075109 27767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.224.67:44063
I20260806 13:07:27.075318 14209 external_mini_cluster.cc:1428] Setting key 12d8ccd88ba4f06f76c4acb28b34f327
I20260806 13:07:27.076064 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 13:07:27.077777 27852 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:27.077307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.13.224.126:37333)
0806 13:07:27.077356 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:27.077357 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:27.077468 (+ 111us) server_negotiation.cc:407] Connection header received
0806 13:07:27.077520 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:27.077521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:27.077530 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:27.077580 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:27.077721 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40366: BlockingRecv error: recv got EOF from 127.0.0.1:40366 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 13:07:27.077826 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 27854
I20260806 13:07:27.085242 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 28068
I20260806 13:07:27.095521 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 28253
I20260806 13:07:27.103230 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 27718
2026-08-06T13:07:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (590 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T13:07:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:07:27Z Disabled control of system clock
I20260806 13:07:27.135895 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:45175
--webserver_interface=127.13.224.126
--webserver_port=0
--builtin_ntp_servers=127.13.224.84:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:45175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 13:07:27.216609 28451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:27.216800 28451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:27.216820 28451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:27.217881 28451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:07:27.217914 28451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:27.217926 28451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:27.217936 28451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:07:27.217947 28451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:07:27.219264 28451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:45175
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:45175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:27.219504 28451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:27.219765 28451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:27.224396 28464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:27.224424 28457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:27.224424 28461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:27.224679 28451 server_base.cc:1034] running on GCE node
I20260806 13:07:27.224906 28451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:27.225184 28451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:27.226334 28451 hybrid_clock.cc:648] HybridClock initialized: now 1786021647226318 us; error 28 us; skew 500 ppm
I20260806 13:07:27.227564 28451 webserver.cc:460] Webserver started at http://127.13.224.126:33273/ using document root <none> and password file <none>
I20260806 13:07:27.227795 28451 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:27.227846 28451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:27.227974 28451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:27.229043 28451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "3cc2a9651f434499b474e7194f8555c3"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "9fa4797bef5146e2e9f77f5a390f6970"
server_key_iv: "084e259d55cada5e54ef6383aec1ed36"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.229370 28451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "3cc2a9651f434499b474e7194f8555c3"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "9fa4797bef5146e2e9f77f5a390f6970"
server_key_iv: "084e259d55cada5e54ef6383aec1ed36"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.230741 28451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:07:27.231421 28471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.231576 28451 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:27.231668 28451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "3cc2a9651f434499b474e7194f8555c3"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "9fa4797bef5146e2e9f77f5a390f6970"
server_key_iv: "084e259d55cada5e54ef6383aec1ed36"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.231727 28451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:27.235204 28451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:27.235448 28451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:27.235574 28451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:27.239276 28451 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:45175
I20260806 13:07:27.239312 28548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:45175 every 8 connection(s)
I20260806 13:07:27.239609 28451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:07:27.239897 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 28451
I20260806 13:07:27.240013 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:07:27.240250 14209 external_mini_cluster.cc:1428] Setting key b58e5351c57b6cc8c3dd55701325435a
I20260806 13:07:27.240404 28551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:27.243393 28554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:27.241057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46232 (local address 127.13.224.126:45175)
0806 13:07:27.241249 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:27.241258 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:27.241291 (+ 33us) server_negotiation.cc:407] Connection header received
0806 13:07:27.241967 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:27.241976 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:27.242080 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:27.242243 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:27.242848 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:27.242851 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:27.242851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:07:27.242856 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:27.242903 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:07:27.242918 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:27.243003 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:27.243148 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 13:07:27.243208 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260806 13:07:27.243875 28551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3: Bootstrap starting.
I20260806 13:07:27.244479 28551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:27.244726 28551 log.cc:824] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:27.245342 28551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3: No bootstrap required, opened a new log
I20260806 13:07:27.246708 28551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc2a9651f434499b474e7194f8555c3" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45175 } }
I20260806 13:07:27.246845 28551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:27.246868 28551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cc2a9651f434499b474e7194f8555c3, State: Initialized, Role: FOLLOWER
I20260806 13:07:27.246959 28551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc2a9651f434499b474e7194f8555c3" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45175 } }
I20260806 13:07:27.247012 28551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:07:27.247035 28551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:07:27.247071 28551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:27.247563 28551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc2a9651f434499b474e7194f8555c3" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45175 } }
I20260806 13:07:27.247655 28551 leader_election.cc:302] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cc2a9651f434499b474e7194f8555c3; no voters:
I20260806 13:07:27.247782 28551 leader_election.cc:288] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:07:27.248013 28551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:07:27.248020 28556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:07:27.248170 28556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [term 1 LEADER]: Becoming Leader. State: Replica: 3cc2a9651f434499b474e7194f8555c3, State: Running, Role: LEADER
I20260806 13:07:27.248292 28556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc2a9651f434499b474e7194f8555c3" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45175 } }
I20260806 13:07:27.248646 28556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cc2a9651f434499b474e7194f8555c3. Latest consensus state: current_term: 1 leader_uuid: "3cc2a9651f434499b474e7194f8555c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc2a9651f434499b474e7194f8555c3" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45175 } } }
I20260806 13:07:27.248728 28556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:27.248654 28558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cc2a9651f434499b474e7194f8555c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cc2a9651f434499b474e7194f8555c3" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 45175 } } }
I20260806 13:07:27.248776 28558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3 [sys.catalog]: This master's current role is: LEADER
I20260806 13:07:27.248924 28569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:07:27.249579 28569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:07:27.251055 28569 catalog_manager.cc:1347] Generated new cluster ID: 225835e0e5a242288040f6d0eb21d559
I20260806 13:07:27.251104 28569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:07:27.267277 28569 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:07:27.267445 28569 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:07:27.270134 28569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cc2a9651f434499b474e7194f8555c3: Generated new TSK 0
I20260806 13:07:27.270290 28569 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:07:27.273643 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:45175
--builtin_ntp_servers=127.13.224.84:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 13:07:27.356724 28582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:27.356916 28582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:27.356945 28582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:27.358116 28582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:27.358155 28582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:27.358187 28582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:07:27.359819 28582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:27.360065 28582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:27.360347 28582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:27.365113 28588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:27.365085 28589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:27.365447 28582 server_base.cc:1034] running on GCE node
W20260806 13:07:27.365617 28592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:27.365806 28582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:27.366065 28582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:27.367219 28582 hybrid_clock.cc:648] HybridClock initialized: now 1786021647367206 us; error 35 us; skew 500 ppm
I20260806 13:07:27.368382 28582 webserver.cc:460] Webserver started at http://127.13.224.65:39915/ using document root <none> and password file <none>
I20260806 13:07:27.368597 28582 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:27.368644 28582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:27.368778 28582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:27.369792 28582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "e07e0dd5076648d28c37ddd75f0df87d"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "07ebbc48c418eb137e8da03b06ff30e6"
server_key_iv: "eac96266630d148348e72286cab98973"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.370149 28582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "e07e0dd5076648d28c37ddd75f0df87d"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "07ebbc48c418eb137e8da03b06ff30e6"
server_key_iv: "eac96266630d148348e72286cab98973"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.371387 28582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:27.372092 28600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.372252 28582 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.372345 28582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "e07e0dd5076648d28c37ddd75f0df87d"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "07ebbc48c418eb137e8da03b06ff30e6"
server_key_iv: "eac96266630d148348e72286cab98973"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.372403 28582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:27.375586 28582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:27.375828 28582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:27.375943 28582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:27.376145 28582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:27.376464 28582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:27.376498 28582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.376516 28582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:27.376528 28582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.381927 28582 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:44465
I20260806 13:07:27.381995 28763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:44465 every 8 connection(s)
I20260806 13:07:27.382267 28582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:07:27.384920 28554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:27.382854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:42715 (local address 127.13.224.126:45175)
0806 13:07:27.382897 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:27.382899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:27.383115 (+ 216us) server_negotiation.cc:407] Connection header received
0806 13:07:27.383551 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:27.383552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:27.383558 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:27.383605 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:27.384668 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:27.384670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:27.384670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:07:27.384676 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:27.384691 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:07:27.384695 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:27.384743 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:27.384837 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 13:07:27.384851 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:07:27.385087 28767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:27.382830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:45175 (local address 127.13.224.65:42715)
0806 13:07:27.382999 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:27.383030 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:27.383456 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:27.383701 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:27.383705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:27.383715 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:27.384527 (+ 812us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:07:27.384528 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:27.384553 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:07:27.384579 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:27.384757 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:27.384764 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:27.384766 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:07:27.384836 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 13:07:27.384896 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260806 13:07:27.385497 28765 heartbeater.cc:347] Connected to a master server at 127.13.224.126:45175
I20260806 13:07:27.385571 28765 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:27.385705 28765 heartbeater.cc:511] Master 127.13.224.126:45175 requested a full tablet report, sending...
I20260806 13:07:27.386183 28495 ts_manager.cc:194] Registered new tserver with Master: e07e0dd5076648d28c37ddd75f0df87d (127.13.224.65:44465)
I20260806 13:07:27.389058 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 28582
I20260806 13:07:27.389142 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:07:27.389333 14209 external_mini_cluster.cc:1428] Setting key 2dc19662ee32c13954a78a112cd51acc
I20260806 13:07:27.390149 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:45175
--builtin_ntp_servers=127.13.224.84:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 13:07:27.470417 28768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:27.470600 28768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:27.470620 28768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:27.471727 28768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:27.471768 28768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:27.471792 28768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:07:27.473099 28768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:27.473313 28768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:27.473582 28768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:27.478251 28776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:27.478252 28779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:27.478345 28777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:27.478432 28768 server_base.cc:1034] running on GCE node
I20260806 13:07:27.478627 28768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:27.478847 28768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:27.479982 28768 hybrid_clock.cc:648] HybridClock initialized: now 1786021647479965 us; error 36 us; skew 500 ppm
I20260806 13:07:27.481161 28768 webserver.cc:460] Webserver started at http://127.13.224.66:40977/ using document root <none> and password file <none>
I20260806 13:07:27.481384 28768 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:27.481432 28768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:27.481559 28768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:27.482606 28768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "8f2af02d4f6f4166a4b46ea5330ae2c8"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "ec66d7533e382948e2ee3d66b01aec42"
server_key_iv: "90dc199b994ca0a84bb0ca8b6afa1cde"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.482965 28768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "8f2af02d4f6f4166a4b46ea5330ae2c8"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "ec66d7533e382948e2ee3d66b01aec42"
server_key_iv: "90dc199b994ca0a84bb0ca8b6afa1cde"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.484135 28768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:07:27.484798 28788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.484941 28768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:27.485013 28768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "8f2af02d4f6f4166a4b46ea5330ae2c8"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "ec66d7533e382948e2ee3d66b01aec42"
server_key_iv: "90dc199b994ca0a84bb0ca8b6afa1cde"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.485075 28768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:27.488260 28768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:27.488482 28768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:27.488597 28768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:27.488790 28768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:27.489066 28768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:27.489099 28768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.489125 28768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:27.489140 28768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.494645 28768 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:44237
I20260806 13:07:27.494688 28964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:44237 every 8 connection(s)
I20260806 13:07:27.494972 28768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:07:27.497443 28554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:27.495580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:39883 (local address 127.13.224.126:45175)
0806 13:07:27.495620 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:27.495622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:27.495838 (+ 216us) server_negotiation.cc:407] Connection header received
0806 13:07:27.496229 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:27.496230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:27.496234 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:27.496287 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:27.497250 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:27.497251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:27.497252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:07:27.497256 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:27.497279 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:07:27.497286 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:27.497326 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:27.497377 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 13:07:27.497388 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:07:27.497586 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:27.495551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:45175 (local address 127.13.224.66:39883)
0806 13:07:27.495725 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:27.495770 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:27.496159 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:27.496335 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:27.496338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:27.496346 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:27.497115 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:07:27.497115 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:27.497140 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:07:27.497166 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:27.497332 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:27.497335 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:27.497337 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:07:27.497378 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 13:07:27.497422 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260806 13:07:27.498085 28965 heartbeater.cc:347] Connected to a master server at 127.13.224.126:45175
I20260806 13:07:27.498171 28965 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:27.498306 28965 heartbeater.cc:511] Master 127.13.224.126:45175 requested a full tablet report, sending...
I20260806 13:07:27.498636 28495 ts_manager.cc:194] Registered new tserver with Master: 8f2af02d4f6f4166a4b46ea5330ae2c8 (127.13.224.66:44237)
I20260806 13:07:27.504925 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 28768
I20260806 13:07:27.505007 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:07:27.505144 14209 external_mini_cluster.cc:1428] Setting key c64cfd7914120362c8c4174c9a30c668
I20260806 13:07:27.505903 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:45175
--builtin_ntp_servers=127.13.224.84:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 13:07:27.585386 28971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:27.585565 28971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:27.585593 28971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:27.586727 28971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:27.586763 28971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:27.586795 28971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:07:27.588109 28971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:27.588338 28971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:27.588608 28971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:27.593210 28981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:27.593344 28983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:27.593254 28980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:27.593380 28971 server_base.cc:1034] running on GCE node
I20260806 13:07:27.593578 28971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:27.593788 28971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:27.594925 28971 hybrid_clock.cc:648] HybridClock initialized: now 1786021647594911 us; error 29 us; skew 500 ppm
I20260806 13:07:27.596065 28971 webserver.cc:460] Webserver started at http://127.13.224.67:42565/ using document root <none> and password file <none>
I20260806 13:07:27.596297 28971 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:27.596356 28971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:27.596485 28971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:27.597445 28971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "3fdc9e8819ae417399438b29522170c2"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "fc4c3303227b3080ed50edbdbb9d0519"
server_key_iv: "d9b9f0d5b75495f87c65d79c295739aa"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.597764 28971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "3fdc9e8819ae417399438b29522170c2"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "fc4c3303227b3080ed50edbdbb9d0519"
server_key_iv: "d9b9f0d5b75495f87c65d79c295739aa"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.598990 28971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:07:27.599696 28992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.599850 28971 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:07:27.599931 28971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "3fdc9e8819ae417399438b29522170c2"
format_stamp: "Formatted at 2026-08-06 13:07:27 on dist-test-slave-q9h9"
server_key: "fc4c3303227b3080ed50edbdbb9d0519"
server_key_iv: "d9b9f0d5b75495f87c65d79c295739aa"
server_key_version: "encryptionkey@0"
I20260806 13:07:27.599996 28971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:27.603272 28971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:27.603498 28971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:27.603615 28971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:27.603793 28971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:27.604056 28971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:27.604089 28971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.604115 28971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:27.604139 28971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:27.609668 28971 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:33959
I20260806 13:07:27.609728 29148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:33959 every 8 connection(s)
I20260806 13:07:27.609998 28971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:07:27.610414 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 28971
I20260806 13:07:27.610498 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:07:27.610687 14209 external_mini_cluster.cc:1428] Setting key d666192908511aaac77ac79791b72f33
I20260806 13:07:27.612685 28554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:27.610677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:42121 (local address 127.13.224.126:45175)
0806 13:07:27.610754 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:27.610756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:27.610960 (+ 204us) server_negotiation.cc:407] Connection header received
0806 13:07:27.611456 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:27.611457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:27.611463 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:27.611510 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:27.612520 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:27.612521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:27.612521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:07:27.612524 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:27.612537 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:07:27.612541 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:27.612582 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:27.612625 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 13:07:27.612635 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 13:07:27.612856 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:27.610658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:45175 (local address 127.13.224.67:42121)
0806 13:07:27.610830 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:27.610868 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:27.611369 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:27.611590 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:27.611594 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:27.611603 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:27.612402 (+ 799us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:07:27.612402 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:27.612428 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:07:27.612452 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:27.612582 (+ 130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:27.612584 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:27.612586 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:07:27.612630 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 13:07:27.612673 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260806 13:07:27.613233 29150 heartbeater.cc:347] Connected to a master server at 127.13.224.126:45175
I20260806 13:07:27.613306 29150 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:27.613443 29150 heartbeater.cc:511] Master 127.13.224.126:45175 requested a full tablet report, sending...
I20260806 13:07:27.613760 28495 ts_manager.cc:194] Registered new tserver with Master: 3fdc9e8819ae417399438b29522170c2 (127.13.224.67:33959)
I20260806 13:07:27.622226 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 13:07:27.623898 28554 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:07:27.623444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46242 (local address 127.13.224.126:45175)
0806 13:07:27.623492 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:27.623494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:27.623601 (+ 107us) server_negotiation.cc:407] Connection header received
0806 13:07:27.623652 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:27.623653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:27.623658 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:27.623702 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:27.623841 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46242: BlockingRecv error: recv got EOF from 127.0.0.1:46242 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 13:07:27.623929 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 28582
I20260806 13:07:27.650349 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 28768
I20260806 13:07:27.656026 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 28971
I20260806 13:07:27.665520 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 28451
2026-08-06T13:07:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (558 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:07:27 dist-test-slave-q9h9 krb5kdc[29165](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:07:27 dist-test-slave-q9h9 krb5kdc[29165](info): set up 2 sockets
Aug 06 13:07:27 dist-test-slave-q9h9 krb5kdc[29165](info): commencing operation
krb5kdc: starting...
W20260806 13:07:28.794516 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:07:28 dist-test-slave-q9h9 krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:07:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:07:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:07:28.902002 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:39649
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:37359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:39649
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:28.980907 29187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:28.981093 29187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:28.981114 29187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:28.982275 29187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:07:28.982311 29187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:28.982323 29187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:28.982337 29187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:07:28.982360 29187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:07:28.983794 29187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:37359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:39649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:39649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:28.984005 29187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:28.984208 29187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:28.988852 29192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:28.988992 29187 server_base.cc:1034] running on GCE node
W20260806 13:07:28.988852 29195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:28.988885 29193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:28.989373 29187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:28.989626 29187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:28.990765 29187 hybrid_clock.cc:648] HybridClock initialized: now 1786021648990749 us; error 29 us; skew 500 ppm
Aug 06 13:07:28 dist-test-slave-q9h9 krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021648, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:28.993054 29187 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:07:28.993465 29187 webserver.cc:460] Webserver started at http://127.13.224.126:40669/ using document root <none> and password file <none>
I20260806 13:07:28.993664 29187 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:28.993710 29187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:28.993841 29187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:28.994864 29187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "d7544898752b4446944da40e1e9c194c"
format_stamp: "Formatted at 2026-08-06 13:07:28 on dist-test-slave-q9h9"
server_key: "996f4907b8a4693dae76071c549060f2"
server_key_iv: "eb2eefcc332b12c83417d28264fb7c17"
server_key_version: "encryptionkey@0"
I20260806 13:07:28.995196 29187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "d7544898752b4446944da40e1e9c194c"
format_stamp: "Formatted at 2026-08-06 13:07:28 on dist-test-slave-q9h9"
server_key: "996f4907b8a4693dae76071c549060f2"
server_key_iv: "eb2eefcc332b12c83417d28264fb7c17"
server_key_version: "encryptionkey@0"
I20260806 13:07:28.996462 29187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:07:28.997099 29206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:28.997269 29187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:28.997354 29187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "d7544898752b4446944da40e1e9c194c"
format_stamp: "Formatted at 2026-08-06 13:07:28 on dist-test-slave-q9h9"
server_key: "996f4907b8a4693dae76071c549060f2"
server_key_iv: "eb2eefcc332b12c83417d28264fb7c17"
server_key_version: "encryptionkey@0"
I20260806 13:07:28.997413 29187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:29.007323 29187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:29.008850 29187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:29.008973 29187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:29.012676 29187 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:39649
I20260806 13:07:29.012713 29283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:39649 every 8 connection(s)
I20260806 13:07:29.013018 29187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:07:29.013926 29284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:29.016021 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 29187
I20260806 13:07:29.016121 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:07:29.016156 29284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c: Bootstrap starting.
I20260806 13:07:29.016315 14209 external_mini_cluster.cc:1428] Setting key b345632d928e4317845c2d367eba4ad8
I20260806 13:07:29.016812 29284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:29.017138 29284 log.cc:824] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:29.017841 29284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c: No bootstrap required, opened a new log
I20260806 13:07:29.020054 29284 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7544898752b4446944da40e1e9c194c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39649 } }
I20260806 13:07:29.020185 29284 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:29.020218 29284 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7544898752b4446944da40e1e9c194c, State: Initialized, Role: FOLLOWER
I20260806 13:07:29.020335 29284 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7544898752b4446944da40e1e9c194c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39649 } }
I20260806 13:07:29.020423 29284 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:07:29.020457 29284 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:07:29.020503 29284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:29.020984 29284 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7544898752b4446944da40e1e9c194c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39649 } }
I20260806 13:07:29.021100 29284 leader_election.cc:302] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7544898752b4446944da40e1e9c194c; no voters:
I20260806 13:07:29.021257 29284 leader_election.cc:288] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:07:29.021348 29291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:07:29.021533 29284 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:07:29.021615 29291 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [term 1 LEADER]: Becoming Leader. State: Replica: d7544898752b4446944da40e1e9c194c, State: Running, Role: LEADER
I20260806 13:07:29.021720 29291 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7544898752b4446944da40e1e9c194c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39649 } }
I20260806 13:07:29.022740 29293 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7544898752b4446944da40e1e9c194c. Latest consensus state: current_term: 1 leader_uuid: "d7544898752b4446944da40e1e9c194c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7544898752b4446944da40e1e9c194c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39649 } } }
I20260806 13:07:29.022825 29293 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [sys.catalog]: This master's current role is: LEADER
Aug 06 13:07:29 dist-test-slave-q9h9 krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:07:29.022985 29292 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7544898752b4446944da40e1e9c194c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7544898752b4446944da40e1e9c194c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39649 } } }
W20260806 13:07:29.023459 29314 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 13:07:29.023933 29292 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c [sys.catalog]: This master's current role is: LEADER
W20260806 13:07:29.023939 29314 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:07:29.024041 29317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:07:29.024199 29317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:07:29.025712 29317 catalog_manager.cc:1347] Generated new cluster ID: 314e4f3aa1534341af007383af0d1e31
I20260806 13:07:29.025760 29317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:07:29.027159 29289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.016837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42378 (local address 127.13.224.126:39649)
0806 13:07:29.017004 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:29.017013 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:29.017068 (+ 55us) server_negotiation.cc:407] Connection header received
0806 13:07:29.017546 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:29.017555 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:29.017625 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:29.017765 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:29.018428 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.019789 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.020541 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.020958 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.021034 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.023835 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:29.023839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:29.023839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:29.023851 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:29.025598 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:29.025962 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:29.025965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:29.025966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:29.026107 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:29.026292 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:29.026293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:29.026294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:29.026588 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:29.026757 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:29.026826 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 13:07:29.026955 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260806 13:07:29.033601 29317 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:07:29.034341 29317 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:07:29.042199 29317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c: Generated new TSK 0
I20260806 13:07:29.042358 29317 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:07:29.070365 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:39649
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:37359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:29.172055 29324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:29.172248 29324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:29.172286 29324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:29.173409 29324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:29.173448 29324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:29.173484 29324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:07:29.174975 29324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:37359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:39649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:29.175206 29324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:29.175407 29324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:29.180104 29336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:29.180068 29334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:29.180157 29324 server_base.cc:1034] running on GCE node
W20260806 13:07:29.180081 29333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:29.180405 29324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:29.180620 29324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:29.181757 29324 hybrid_clock.cc:648] HybridClock initialized: now 1786021649181744 us; error 33 us; skew 500 ppm
Aug 06 13:07:29 dist-test-slave-q9h9 krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021649, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:29.184053 29324 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:07:29.184427 29324 webserver.cc:460] Webserver started at http://127.13.224.65:37257/ using document root <none> and password file <none>
I20260806 13:07:29.184594 29324 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:29.184633 29324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:29.184719 29324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:29.185683 29324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "6dfc542cee764716baff3ec04d3a2a5d"
format_stamp: "Formatted at 2026-08-06 13:07:29 on dist-test-slave-q9h9"
server_key: "497c470aa6f77dd94cea86fe24b03b0f"
server_key_iv: "8229d42a18707fd7d6582b4b19758a4a"
server_key_version: "encryptionkey@0"
I20260806 13:07:29.186138 29324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "6dfc542cee764716baff3ec04d3a2a5d"
format_stamp: "Formatted at 2026-08-06 13:07:29 on dist-test-slave-q9h9"
server_key: "497c470aa6f77dd94cea86fe24b03b0f"
server_key_iv: "8229d42a18707fd7d6582b4b19758a4a"
server_key_version: "encryptionkey@0"
I20260806 13:07:29.187316 29324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:29.187937 29346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:29.188077 29324 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:29.188159 29324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "6dfc542cee764716baff3ec04d3a2a5d"
format_stamp: "Formatted at 2026-08-06 13:07:29 on dist-test-slave-q9h9"
server_key: "497c470aa6f77dd94cea86fe24b03b0f"
server_key_iv: "8229d42a18707fd7d6582b4b19758a4a"
server_key_version: "encryptionkey@0"
I20260806 13:07:29.188226 29324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:29.229746 29324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:29.231410 29324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:29.231549 29324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:29.231760 29324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:29.232059 29324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:29.232100 29324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:29.232128 29324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:29.232146 29324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:29.237560 29324 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:43957
I20260806 13:07:29.237620 29519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:43957 every 8 connection(s)
I20260806 13:07:29.237890 29324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
Aug 06 13:07:29 dist-test-slave-q9h9 krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021649, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:07:29.245962 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 29324
I20260806 13:07:29.246064 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:07:29.246260 14209 external_mini_cluster.cc:1428] Setting key 63566d208cdd57f366c0acd40e9a1125
I20260806 13:07:29.246366 29289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.238616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:59351 (local address 127.13.224.126:39649)
0806 13:07:29.238664 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:29.238666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:29.238856 (+ 190us) server_negotiation.cc:407] Connection header received
0806 13:07:29.239357 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:29.239359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:29.239371 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:29.239417 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:29.240326 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.240815 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.241785 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.241987 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.242068 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.244066 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:29.244068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:29.244068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:29.244075 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:29.245128 (+ 1053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:29.245461 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:29.245462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:29.245462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:29.245506 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:29.245687 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:29.245688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:29.245688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:29.245784 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:29.245844 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:29.246187 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 13:07:29.246277 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:07:29.246507 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.238556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39649 (local address 127.13.224.65:59351)
0806 13:07:29.238736 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:29.238774 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:29.239260 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:29.239499 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:29.239503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:29.240036 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:29.240219 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.240224 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.240898 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.240899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.241690 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.241693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.242076 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.242077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.242208 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.242645 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:29.242654 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:29.243984 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:29.245197 (+ 1213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:29.245204 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:29.245206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:29.245406 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:29.245550 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:29.245550 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:29.245551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:29.245647 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:29.245858 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:29.245865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:29.245989 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 13:07:29.246148 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 13:07:29.246299 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260806 13:07:29.246958 29520 heartbeater.cc:347] Connected to a master server at 127.13.224.126:39649
I20260806 13:07:29.247062 29520 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:29.247259 29520 heartbeater.cc:511] Master 127.13.224.126:39649 requested a full tablet report, sending...
I20260806 13:07:29.247761 29226 ts_manager.cc:194] Registered new tserver with Master: 6dfc542cee764716baff3ec04d3a2a5d (127.13.224.65:43957)
I20260806 13:07:29.248354 29226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:59351
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:07:29.268666 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:39649
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:37359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:29.352342 29528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:29.352558 29528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:29.352587 29528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:29.353782 29528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:29.353823 29528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:29.353858 29528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:07:29.355366 29528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:37359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:39649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:29.355620 29528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:29.355834 29528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:29.361050 29538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:29.361177 29537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:29.361047 29540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:29.361178 29528 server_base.cc:1034] running on GCE node
I20260806 13:07:29.361492 29528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:29.361719 29528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:29.362870 29528 hybrid_clock.cc:648] HybridClock initialized: now 1786021649362854 us; error 34 us; skew 500 ppm
Aug 06 13:07:29 dist-test-slave-q9h9 krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021649, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:29.365224 29528 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:07:29.365633 29528 webserver.cc:460] Webserver started at http://127.13.224.66:38831/ using document root <none> and password file <none>
I20260806 13:07:29.365851 29528 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:29.365895 29528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:29.366034 29528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:29.367049 29528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "92f01006bccf43e8a9c08feed3316445"
format_stamp: "Formatted at 2026-08-06 13:07:29 on dist-test-slave-q9h9"
server_key: "559800fdf0586069a3df4b2e629840fc"
server_key_iv: "2887518308b8ec87c3f79fdb39dabb5e"
server_key_version: "encryptionkey@0"
I20260806 13:07:29.367373 29528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "92f01006bccf43e8a9c08feed3316445"
format_stamp: "Formatted at 2026-08-06 13:07:29 on dist-test-slave-q9h9"
server_key: "559800fdf0586069a3df4b2e629840fc"
server_key_iv: "2887518308b8ec87c3f79fdb39dabb5e"
server_key_version: "encryptionkey@0"
I20260806 13:07:29.368603 29528 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:07:29.369352 29549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:29.369506 29528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:07:29.369583 29528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "92f01006bccf43e8a9c08feed3316445"
format_stamp: "Formatted at 2026-08-06 13:07:29 on dist-test-slave-q9h9"
server_key: "559800fdf0586069a3df4b2e629840fc"
server_key_iv: "2887518308b8ec87c3f79fdb39dabb5e"
server_key_version: "encryptionkey@0"
I20260806 13:07:29.369649 29528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:29.388895 29528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:29.391036 29528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:29.391165 29528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:29.391381 29528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:29.391686 29528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:29.391719 29528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:29.391748 29528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:29.391764 29528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:29.397338 29528 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:41545
I20260806 13:07:29.397393 29725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:41545 every 8 connection(s)
I20260806 13:07:29.397693 29528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
Aug 06 13:07:29 dist-test-slave-q9h9 krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021649, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:07:29.404062 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 29528
I20260806 13:07:29.404172 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:07:29.404368 14209 external_mini_cluster.cc:1428] Setting key 7fb22ad7da724a4389f5610448b26ad6
I20260806 13:07:29.406792 29289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.398391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:34963 (local address 127.13.224.126:39649)
0806 13:07:29.398479 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:29.398481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:29.398644 (+ 163us) server_negotiation.cc:407] Connection header received
0806 13:07:29.399106 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:29.399107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:29.399116 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:29.399163 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:29.400107 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.400661 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.401634 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.401845 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.401910 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.404006 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:29.404008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:29.404009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:29.404017 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:29.405493 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:29.405884 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:29.405886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:29.405886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:29.405949 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:29.406212 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:29.406213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:29.406213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:29.406343 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:29.406402 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:29.406624 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 13:07:29.406706 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 13:07:29.406929 29728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.398366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39649 (local address 127.13.224.66:34963)
0806 13:07:29.398526 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:29.398563 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:29.399030 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:29.399224 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:29.399227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:29.399804 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:29.400028 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.400033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.400734 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.400734 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.401530 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.401533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.401915 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.401915 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.402011 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.402507 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:29.402515 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:29.403902 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:29.405579 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:29.405586 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:29.405588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:29.405817 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:29.406005 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:29.406006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:29.406006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:29.406156 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:29.406401 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:29.406405 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:29.406517 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 13:07:29.406593 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 13:07:29.406727 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260806 13:07:29.407387 29726 heartbeater.cc:347] Connected to a master server at 127.13.224.126:39649
I20260806 13:07:29.407472 29726 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:29.407645 29726 heartbeater.cc:511] Master 127.13.224.126:39649 requested a full tablet report, sending...
I20260806 13:07:29.408008 29226 ts_manager.cc:194] Registered new tserver with Master: 92f01006bccf43e8a9c08feed3316445 (127.13.224.66:41545)
I20260806 13:07:29.408360 29226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:34963
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:07:29.428179 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:39649
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:37359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:07:29.507396 29733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:07:29.507579 29733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:07:29.507602 29733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:07:29.508677 29733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:07:29.508715 29733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:07:29.508740 29733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:07:29.510205 29733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:37359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:39649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:07:29.510397 29733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:07:29.510599 29733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:07:29.515246 29744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:29.515259 29743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:07:29.515260 29748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:07:29.515599 29733 server_base.cc:1034] running on GCE node
I20260806 13:07:29.515779 29733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:07:29.515982 29733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:07:29.517130 29733 hybrid_clock.cc:648] HybridClock initialized: now 1786021649517124 us; error 40 us; skew 500 ppm
Aug 06 13:07:29 dist-test-slave-q9h9 krb5kdc[29165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021649, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:07:29.519564 29733 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:07:29.519933 29733 webserver.cc:460] Webserver started at http://127.13.224.67:44665/ using document root <none> and password file <none>
I20260806 13:07:29.520099 29733 fs_manager.cc:359] Metadata directory not provided
I20260806 13:07:29.520138 29733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:07:29.520223 29733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:07:29.521195 29733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e"
format_stamp: "Formatted at 2026-08-06 13:07:29 on dist-test-slave-q9h9"
server_key: "3309349d593a73ef1b0b3df036e92cf1"
server_key_iv: "df3b5b8dba602c7c356f1f1e745109af"
server_key_version: "encryptionkey@0"
I20260806 13:07:29.521483 29733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e"
format_stamp: "Formatted at 2026-08-06 13:07:29 on dist-test-slave-q9h9"
server_key: "3309349d593a73ef1b0b3df036e92cf1"
server_key_iv: "df3b5b8dba602c7c356f1f1e745109af"
server_key_version: "encryptionkey@0"
I20260806 13:07:29.522868 29733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:07:29.523502 29763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:29.523630 29733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:07:29.523701 29733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e"
format_stamp: "Formatted at 2026-08-06 13:07:29 on dist-test-slave-q9h9"
server_key: "3309349d593a73ef1b0b3df036e92cf1"
server_key_iv: "df3b5b8dba602c7c356f1f1e745109af"
server_key_version: "encryptionkey@0"
I20260806 13:07:29.523747 29733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:07:29.553910 29733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:07:29.555856 29733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:07:29.555959 29733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:07:29.556130 29733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:07:29.556366 29733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:07:29.556393 29733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:29.556413 29733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:07:29.556429 29733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:07:29.561700 29733 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:41545
I20260806 13:07:29.561771 29942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:41545 every 8 connection(s)
I20260806 13:07:29.562006 29733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:07:29.562611 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 29733
I20260806 13:07:29.562698 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:07:29.562865 14209 external_mini_cluster.cc:1428] Setting key 19231eb7731059c5312117da1cc306db
Aug 06 13:07:29 dist-test-slave-q9h9 krb5kdc[29165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021649, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:07:29.569989 29289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.562708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:40781 (local address 127.13.224.126:39649)
0806 13:07:29.562758 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:29.562760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:29.562955 (+ 195us) server_negotiation.cc:407] Connection header received
0806 13:07:29.563456 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:29.563457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:29.563470 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:29.563519 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:29.564386 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.564874 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.565807 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.566009 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.566091 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.567995 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:29.567996 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:29.567996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:29.568003 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:29.569030 (+ 1027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:29.569350 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:29.569351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:29.569351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:29.569394 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:29.569557 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:29.569557 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:29.569557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:29.569645 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:29.569687 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:29.569874 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 13:07:29.569927 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 13:07:29.570219 29947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.562670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39649 (local address 127.13.224.67:40781)
0806 13:07:29.562834 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:29.562873 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:29.563362 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:29.563585 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:29.563589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:29.564117 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:07:29.564300 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.564305 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.564953 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.564955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.565727 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.565730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.566121 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.566122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.566238 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.566675 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:07:29.566682 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:07:29.567921 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:07:29.569097 (+ 1176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:29.569104 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:29.569105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:29.569296 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:29.569435 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:07:29.569435 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:07:29.569436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:07:29.569518 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:07:29.569684 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:07:29.569687 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:07:29.569786 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 13:07:29.569858 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 13:07:29.569964 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260806 13:07:29.570627 29943 heartbeater.cc:347] Connected to a master server at 127.13.224.126:39649
I20260806 13:07:29.570713 29943 heartbeater.cc:464] Registering TS with master...
I20260806 13:07:29.570873 29943 heartbeater.cc:511] Master 127.13.224.126:39649 requested a full tablet report, sending...
I20260806 13:07:29.571201 29226 ts_manager.cc:194] Registered new tserver with Master: b3d3f6ee4af14cfe89aa1fc8046b984e (127.13.224.67:41545)
I20260806 13:07:29.571564 29226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:40781
I20260806 13:07:29.574292 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:07:29.580816 29289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.575462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42394 (local address 127.13.224.126:39649)
0806 13:07:29.575517 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:29.575519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:29.575614 (+ 95us) server_negotiation.cc:407] Connection header received
0806 13:07:29.575672 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:29.575673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:29.575682 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:29.575725 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:07:29.576237 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.576677 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.577369 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.577548 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.577598 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.578658 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:07:29.578659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:07:29.578660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:07:29.578666 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:07:29.579832 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:29.580121 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:29.580122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:29.580123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:29.580171 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:07:29.580327 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:07:29.580327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:07:29.580328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:07:29.580457 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:07:29.580539 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:29.580670 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 13:07:29.580736 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 13:07:29.582173 29226 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:07:29.583671 29226 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:07:29.589463 29958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.585645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:41545 (local address 127.0.0.1:51964)
0806 13:07:29.585899 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:29.585936 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:29.586008 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:29.586337 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:29.586340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:29.586342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:29.586399 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.586400 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.587316 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.587317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.588272 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.588275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.589220 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.589221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.589285 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.589295 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 13:07:29.589346 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 13:07:29.589394 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":60,"threads_started":1}
I20260806 13:07:29.589483 29963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.585703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51964 (local address 127.13.224.67:41545)
0806 13:07:29.585877 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:29.585882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:29.586004 (+ 122us) server_negotiation.cc:407] Connection header received
0806 13:07:29.586157 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:29.586161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:29.586176 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:29.586226 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:29.586482 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.587222 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.588415 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.589104 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.589137 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.589176 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:29.589379 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 13:07:29.589414 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260806 13:07:29.589818 29961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.585645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:43957 (local address 127.0.0.1:36764)
0806 13:07:29.586106 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:29.586115 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:29.586166 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:29.586414 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:29.586415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:29.586418 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:29.586574 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.586576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.587882 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.587883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.588752 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.588755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.589594 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.589595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.589656 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.589660 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:07:29.589714 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 13:07:29.589750 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":22,"thread_start_us":101,"threads_started":1}
I20260806 13:07:29.590109 29966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.585898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36764 (local address 127.13.224.65:43957)
0806 13:07:29.586103 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:29.586108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:29.586158 (+ 50us) server_negotiation.cc:407] Connection header received
0806 13:07:29.586232 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:29.586234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:29.586249 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:29.586332 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:29.586654 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.587759 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.588820 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.589445 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.589508 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.589557 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:29.589846 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 13:07:29.590006 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":85,"threads_started":1}
I20260806 13:07:29.590197 29956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.585650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:41545 (local address 127.0.0.1:49530)
0806 13:07:29.586255 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:29.586262 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:29.586299 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:29.586544 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:29.586545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:29.586547 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:29.586592 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.586593 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.587858 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.587860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.588970 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.588972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.589901 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.589902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.589960 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.589964 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:07:29.590068 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 13:07:29.590135 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":27,"thread_start_us":95,"threads_started":1}
I20260806 13:07:29.590226 29968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.586124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49530 (local address 127.13.224.66:41545)
0806 13:07:29.586303 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:29.586307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:29.586318 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:07:29.586355 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:29.586357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:29.586371 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:29.586463 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:29.586666 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.587759 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.589050 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.589760 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.589815 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.589867 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:29.590070 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 13:07:29.590119 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":75,"threads_started":1}
I20260806 13:07:29.590325 29840 tablet_service.cc:1467] Processing CreateTablet for tablet 793d320479cf4c18adedc1e7d2a303cf (DEFAULT_TABLE table=test-table [id=6c454d6678fe49eeae5e9cd9a7efeb5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:29.590678 29840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793d320479cf4c18adedc1e7d2a303cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:29.590843 29627 tablet_service.cc:1467] Processing CreateTablet for tablet 793d320479cf4c18adedc1e7d2a303cf (DEFAULT_TABLE table=test-table [id=6c454d6678fe49eeae5e9cd9a7efeb5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:29.591168 29627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793d320479cf4c18adedc1e7d2a303cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:29.593173 29970 tablet_bootstrap.cc:492] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e: Bootstrap starting.
I20260806 13:07:29.593565 29972 tablet_bootstrap.cc:492] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445: Bootstrap starting.
I20260806 13:07:29.593936 29970 tablet_bootstrap.cc:654] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:29.594120 29420 tablet_service.cc:1467] Processing CreateTablet for tablet 793d320479cf4c18adedc1e7d2a303cf (DEFAULT_TABLE table=test-table [id=6c454d6678fe49eeae5e9cd9a7efeb5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:07:29.594447 29420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793d320479cf4c18adedc1e7d2a303cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:07:29.594491 29970 log.cc:824] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:29.595237 29970 tablet_bootstrap.cc:492] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e: No bootstrap required, opened a new log
I20260806 13:07:29.595322 29970 ts_tablet_manager.cc:1397] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:07:29.597148 29970 raft_consensus.cc:348] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41545 } } peers { permanent_uuid: "92f01006bccf43e8a9c08feed3316445" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 41545 } } peers { permanent_uuid: "6dfc542cee764716baff3ec04d3a2a5d" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43957 } }
I20260806 13:07:29.597318 29970 raft_consensus.cc:374] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:29.597368 29970 raft_consensus.cc:729] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3d3f6ee4af14cfe89aa1fc8046b984e, State: Initialized, Role: FOLLOWER
I20260806 13:07:29.597486 29970 consensus_queue.cc:260] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41545 } } peers { permanent_uuid: "92f01006bccf43e8a9c08feed3316445" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 41545 } } peers { permanent_uuid: "6dfc542cee764716baff3ec04d3a2a5d" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43957 } }
I20260806 13:07:29.597784 29970 ts_tablet_manager.cc:1428] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 13:07:29.597810 29943 heartbeater.cc:503] Master 127.13.224.126:39649 was elected leader, sending a full tablet report...
I20260806 13:07:29.597970 29975 tablet_bootstrap.cc:492] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d: Bootstrap starting.
I20260806 13:07:29.598083 29972 tablet_bootstrap.cc:654] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:29.598632 29975 tablet_bootstrap.cc:654] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d: Neither blocks nor log segments found. Creating new log.
I20260806 13:07:29.598620 29972 log.cc:824] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:29.599152 29975 log.cc:824] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d: Log is configured to *not* fsync() on all Append() calls
I20260806 13:07:29.599473 29972 tablet_bootstrap.cc:492] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445: No bootstrap required, opened a new log
I20260806 13:07:29.599527 29972 ts_tablet_manager.cc:1397] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260806 13:07:29.599884 29975 tablet_bootstrap.cc:492] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d: No bootstrap required, opened a new log
I20260806 13:07:29.599958 29975 ts_tablet_manager.cc:1397] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 13:07:29.601419 29972 raft_consensus.cc:348] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41545 } } peers { permanent_uuid: "92f01006bccf43e8a9c08feed3316445" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 41545 } } peers { permanent_uuid: "6dfc542cee764716baff3ec04d3a2a5d" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43957 } }
I20260806 13:07:29.601579 29972 raft_consensus.cc:374] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:29.601615 29972 raft_consensus.cc:729] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f01006bccf43e8a9c08feed3316445, State: Initialized, Role: FOLLOWER
I20260806 13:07:29.601723 29972 consensus_queue.cc:260] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41545 } } peers { permanent_uuid: "92f01006bccf43e8a9c08feed3316445" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 41545 } } peers { permanent_uuid: "6dfc542cee764716baff3ec04d3a2a5d" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43957 } }
I20260806 13:07:29.601770 29975 raft_consensus.cc:348] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41545 } } peers { permanent_uuid: "92f01006bccf43e8a9c08feed3316445" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 41545 } } peers { permanent_uuid: "6dfc542cee764716baff3ec04d3a2a5d" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43957 } }
I20260806 13:07:29.601910 29975 raft_consensus.cc:374] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:07:29.601956 29975 raft_consensus.cc:729] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dfc542cee764716baff3ec04d3a2a5d, State: Initialized, Role: FOLLOWER
I20260806 13:07:29.602005 29972 ts_tablet_manager.cc:1428] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 13:07:29.602072 29726 heartbeater.cc:503] Master 127.13.224.126:39649 was elected leader, sending a full tablet report...
I20260806 13:07:29.602095 29975 consensus_queue.cc:260] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41545 } } peers { permanent_uuid: "92f01006bccf43e8a9c08feed3316445" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 41545 } } peers { permanent_uuid: "6dfc542cee764716baff3ec04d3a2a5d" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43957 } }
I20260806 13:07:29.602344 29975 ts_tablet_manager.cc:1428] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 13:07:29.602391 29520 heartbeater.cc:503] Master 127.13.224.126:39649 was elected leader, sending a full tablet report...
W20260806 13:07:29.648216 29727 tablet.cc:2369] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 13:07:29.738600 29521 tablet.cc:2369] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:07:29.744223 29976 raft_consensus.cc:482] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:07:29.744323 29976 raft_consensus.cc:504] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41545 } } peers { permanent_uuid: "92f01006bccf43e8a9c08feed3316445" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 41545 } } peers { permanent_uuid: "6dfc542cee764716baff3ec04d3a2a5d" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43957 } }
I20260806 13:07:29.744676 29976 leader_election.cc:288] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92f01006bccf43e8a9c08feed3316445 (127.13.224.66:41545), 6dfc542cee764716baff3ec04d3a2a5d (127.13.224.65:43957)
I20260806 13:07:29.747460 29966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.744944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:48259 (local address 127.13.224.65:43957)
0806 13:07:29.744981 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:29.744983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:29.745160 (+ 177us) server_negotiation.cc:407] Connection header received
0806 13:07:29.745197 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:29.745198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:29.745208 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:29.745256 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:29.745398 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.745857 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.746805 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.747154 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.747216 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.747233 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:29.747362 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 13:07:29.747396 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:07:29.747483 29968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.744910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:49717 (local address 127.13.224.66:41545)
0806 13:07:29.744975 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:29.744977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:29.744989 (+ 12us) server_negotiation.cc:407] Connection header received
0806 13:07:29.745017 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:29.745018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:29.745027 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:29.745081 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:07:29.745236 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.745707 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.746722 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.747161 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.747227 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.747236 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:29.747380 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 13:07:29.747415 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 13:07:29.747500 29947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.744858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:41545 (local address 127.13.224.67:49717)
0806 13:07:29.744904 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:29.744916 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:29.744957 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:29.745102 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:29.745103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:29.745106 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:29.745175 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.745176 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.745792 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.745793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.746632 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.746634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.747234 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.747235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.747318 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.747321 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:07:29.747366 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 13:07:29.747407 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":57}
I20260806 13:07:29.747511 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.744858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:43957 (local address 127.13.224.67:48259)
0806 13:07:29.745087 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:29.745095 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:07:29.745146 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:07:29.745265 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:07:29.745266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:07:29.745268 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:07:29.745341 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.745342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.745932 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.745933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.746703 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:07:29.746706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.747227 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.747228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:07:29.747300 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.747304 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:07:29.747348 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 13:07:29.747390 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":47,"thread_start_us":115,"threads_started":1}
I20260806 13:07:29.747845 29659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793d320479cf4c18adedc1e7d2a303cf" candidate_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f01006bccf43e8a9c08feed3316445" is_pre_election: true
I20260806 13:07:29.747896 29442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793d320479cf4c18adedc1e7d2a303cf" candidate_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dfc542cee764716baff3ec04d3a2a5d" is_pre_election: true
I20260806 13:07:29.748000 29659 raft_consensus.cc:2454] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3d3f6ee4af14cfe89aa1fc8046b984e in term 0.
I20260806 13:07:29.748060 29442 raft_consensus.cc:2454] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3d3f6ee4af14cfe89aa1fc8046b984e in term 0.
I20260806 13:07:29.748224 29767 leader_election.cc:302] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92f01006bccf43e8a9c08feed3316445, b3d3f6ee4af14cfe89aa1fc8046b984e; no voters:
I20260806 13:07:29.748396 29976 raft_consensus.cc:2781] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:07:29.748436 29976 raft_consensus.cc:482] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:07:29.748457 29976 raft_consensus.cc:3037] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:29.749153 29976 raft_consensus.cc:504] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41545 } } peers { permanent_uuid: "92f01006bccf43e8a9c08feed3316445" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 41545 } } peers { permanent_uuid: "6dfc542cee764716baff3ec04d3a2a5d" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43957 } }
I20260806 13:07:29.749269 29976 leader_election.cc:288] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [CANDIDATE]: Term 1 election: Requested vote from peers 92f01006bccf43e8a9c08feed3316445 (127.13.224.66:41545), 6dfc542cee764716baff3ec04d3a2a5d (127.13.224.65:43957)
I20260806 13:07:29.749433 29659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793d320479cf4c18adedc1e7d2a303cf" candidate_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f01006bccf43e8a9c08feed3316445"
I20260806 13:07:29.749483 29442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793d320479cf4c18adedc1e7d2a303cf" candidate_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dfc542cee764716baff3ec04d3a2a5d"
I20260806 13:07:29.749519 29659 raft_consensus.cc:3037] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:29.749547 29442 raft_consensus.cc:3037] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:07:29.750447 29442 raft_consensus.cc:2454] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3d3f6ee4af14cfe89aa1fc8046b984e in term 1.
I20260806 13:07:29.750447 29659 raft_consensus.cc:2454] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3d3f6ee4af14cfe89aa1fc8046b984e in term 1.
I20260806 13:07:29.750607 29767 leader_election.cc:302] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92f01006bccf43e8a9c08feed3316445, b3d3f6ee4af14cfe89aa1fc8046b984e; no voters:
I20260806 13:07:29.750699 29976 raft_consensus.cc:2781] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:07:29.750895 29976 raft_consensus.cc:686] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [term 1 LEADER]: Becoming Leader. State: Replica: b3d3f6ee4af14cfe89aa1fc8046b984e, State: Running, Role: LEADER
I20260806 13:07:29.750983 29976 consensus_queue.cc:237] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41545 } } peers { permanent_uuid: "92f01006bccf43e8a9c08feed3316445" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 41545 } } peers { permanent_uuid: "6dfc542cee764716baff3ec04d3a2a5d" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43957 } }
I20260806 13:07:29.751801 29228 catalog_manager.cc:5495] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e reported cstate change: term changed from 0 to 1, leader changed from <none> to b3d3f6ee4af14cfe89aa1fc8046b984e (127.13.224.67). New cstate: current_term: 1 leader_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3d3f6ee4af14cfe89aa1fc8046b984e" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92f01006bccf43e8a9c08feed3316445" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dfc542cee764716baff3ec04d3a2a5d" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43957 } health_report { overall_health: UNKNOWN } } }
I20260806 13:07:29.767776 29963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.765724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51974 (local address 127.13.224.67:41545)
0806 13:07:29.765779 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 13:07:29.765781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:07:29.765788 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:07:29.765813 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:07:29.765814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:07:29.765828 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:07:29.765884 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:07:29.766129 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.766553 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.767233 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:07:29.767398 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:07:29.767438 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:07:29.767539 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:07:29.767623 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:07:29.767672 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 13:07:29.767685 (+ 13us) server_negotiation.cc:299] Negotiation successful
0806 13:07:29.767699 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 13:07:29.769287 29659 raft_consensus.cc:1261] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445 [term 1 FOLLOWER]: Refusing update from remote peer b3d3f6ee4af14cfe89aa1fc8046b984e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:29.769294 29442 raft_consensus.cc:1261] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d [term 1 FOLLOWER]: Refusing update from remote peer b3d3f6ee4af14cfe89aa1fc8046b984e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:07:29.769599 29983 consensus_queue.cc:1035] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [LEADER]: Connected to new peer: Peer: permanent_uuid: "92f01006bccf43e8a9c08feed3316445" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:29.769717 29976 consensus_queue.cc:1035] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dfc542cee764716baff3ec04d3a2a5d" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:07:29.771844 29987 mvcc.cc:205] Tried to move back new op lower bound from 7315544677452959744 to 7315544677380689920. Current Snapshot: MvccSnapshot[applied={T|T < 7315544677452959744}]
I20260806 13:07:29.774415 29228 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:29.774530 29228 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:07:29.775211 29228 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d7544898752b4446944da40e1e9c194c: Sending DeleteTablet for 3 replicas of tablet 793d320479cf4c18adedc1e7d2a303cf
I20260806 13:07:29.775513 29840 tablet_service.cc:1514] Processing DeleteTablet for tablet 793d320479cf4c18adedc1e7d2a303cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:29 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:51964
I20260806 13:07:29.775647 29420 tablet_service.cc:1514] Processing DeleteTablet for tablet 793d320479cf4c18adedc1e7d2a303cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:29 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:36764
I20260806 13:07:29.775730 29627 tablet_service.cc:1514] Processing DeleteTablet for tablet 793d320479cf4c18adedc1e7d2a303cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:07:29 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:49530
I20260806 13:07:29.775839 30002 tablet_replica.cc:330] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e: stopping tablet replica
I20260806 13:07:29.775889 30003 tablet_replica.cc:330] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d: stopping tablet replica
I20260806 13:07:29.775985 30003 raft_consensus.cc:2229] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:29.776005 30002 raft_consensus.cc:2229] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:07:29.776093 30003 raft_consensus.cc:2258] T 793d320479cf4c18adedc1e7d2a303cf P 6dfc542cee764716baff3ec04d3a2a5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:29.776163 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 29324
I20260806 13:07:29.778468 30002 raft_consensus.cc:2258] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:29.781373 30002 ts_tablet_manager.cc:1905] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:29.783277 30002 ts_tablet_manager.cc:1918] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:07:29.783411 30002 log.cc:1196] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/793d320479cf4c18adedc1e7d2a303cf
I20260806 13:07:29.783780 30002 ts_tablet_manager.cc:1939] T 793d320479cf4c18adedc1e7d2a303cf P b3d3f6ee4af14cfe89aa1fc8046b984e: Deleting consensus metadata
I20260806 13:07:29.784263 29207 catalog_manager.cc:4841] TS b3d3f6ee4af14cfe89aa1fc8046b984e (127.13.224.67:41545): tablet 793d320479cf4c18adedc1e7d2a303cf (table test-table [id=6c454d6678fe49eeae5e9cd9a7efeb5e]) successfully deleted
I20260806 13:07:29.801393 30006 tablet_replica.cc:330] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445: stopping tablet replica
I20260806 13:07:29.801622 30006 raft_consensus.cc:2229] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:07:29.801807 30006 raft_consensus.cc:2258] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:07:29.802448 30006 ts_tablet_manager.cc:1905] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:07:29.804059 30006 ts_tablet_manager.cc:1918] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:07:29.804152 30006 log.cc:1196] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/793d320479cf4c18adedc1e7d2a303cf
I20260806 13:07:29.804431 30006 ts_tablet_manager.cc:1939] T 793d320479cf4c18adedc1e7d2a303cf P 92f01006bccf43e8a9c08feed3316445: Deleting consensus metadata
I20260806 13:07:29.804915 29209 catalog_manager.cc:4841] TS 92f01006bccf43e8a9c08feed3316445 (127.13.224.66:41545): tablet 793d320479cf4c18adedc1e7d2a303cf (table test-table [id=6c454d6678fe49eeae5e9cd9a7efeb5e]) successfully deleted
W20260806 13:07:29.806843 29210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:43957 (error 108)
I20260806 13:07:29.807355 29956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:07:29.807223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:43957 (local address 127.0.0.1:36768)
0806 13:07:29.807281 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 13:07:29.807320 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:43957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 13:07:29.807380 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 29528
W20260806 13:07:29.822841 29210 catalog_manager.cc:4584] TS 6dfc542cee764716baff3ec04d3a2a5d (127.13.224.65:43957): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 793d320479cf4c18adedc1e7d2a303cf: Network error: Client connection negotiation failed: client connection to 127.13.224.65:43957: connect: Connection refused (error 111)
I20260806 13:07:29.827754 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 29733
I20260806 13:07:29.840714 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 29187
2026-08-06T13:07:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2180 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:07:29 dist-test-slave-q9h9 krb5kdc[30015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:07:29 dist-test-slave-q9h9 krb5kdc[30015](info): set up 2 sockets
Aug 06 13:07:29 dist-test-slave-q9h9 krb5kdc[30015](info): commencing operation
krb5kdc: starting...
W20260806 13:07:30.954270 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:07:31 dist-test-slave-q9h9 krb5kdc[30015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:07:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:07:31Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/rangeradmin_127.13.224.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/rangeradmin_127.13.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/rangerlookup_127.13.224.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/rangerlookup_127.13.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/HTTP_127.13.224.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/HTTP_127.13.224.84@KRBTEST.COM.keytab.
I20260806 13:07:31.101154 14209 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXc7v3G/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/postgres -l logfile start
2026-08-06 13:07:33.336 UTC [30062] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 13:07:33.336 UTC [30062] LOG: listening on IPv4 address "127.13.224.84", port 34979
2026-08-06 13:07:33.339 UTC [30062] LOG: listening on Unix socket "/tmp/.s.PGSQL.34979"
2026-08-06 13:07:33.358 UTC [30067] LOG: database system was shut down at 2026-08-06 13:07:32 UTC
2026-08-06 13:07:33.361 UTC [30062] LOG: database system is ready to accept connections
I20260806 13:07:34.406562 14209 mini_postgres.cc:96] Postgres bound to 34979
2026-08-06 13:07:34.414 UTC [30078] FATAL: database "slave" does not exist
127.13.224.84:34979 - accepting connections
I20260806 13:07:34.414997 14209 mini_ranger.cc:161] Starting Ranger...
I20260806 13:07:34.430526 14209 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 13:07:34.836324 14209 mini_ranger.cc:87] Created ranger Postgres database
I20260806 13:07:34.836393 14209 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-admin
2026-08-06 13:07:35,157 [I] DB FLAVOR :POSTGRES
2026-08-06 13:07:35,157 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 13:07:35,157 [I] Checking connection..
2026-08-06 13:07:35,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 13:07:35,422 [I] Checking connection passed.
2026-08-06 13:07:35,422 [I] --------- Verifying version history table ---------
2026-08-06 13:07:35,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 13:07:35,680 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 13:07:35,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 13:07:35,923 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 13:07:35,923 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 13:07:35,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 13:07:36.171 UTC [30172] WARNING: there is no transaction in progress
2026-08-06 13:07:36,180 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 13:07:36,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 13:07:36,427 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 13:07:36,428 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 13:07:36,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 13:07:36,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 13:07:36,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-q9h9.c.gcp-upstream.internal','N') ;"
2026-08-06 13:07:37,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 13:07:37,395 [I] Table x_portal_user does not exist in database ranger
2026-08-06 13:07:37,395 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 13:07:37,656 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 13:07:37,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 13:07:37,916 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 13:07:38,172 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 13:07:38,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 13:07:39.064 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.080 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.095 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.109 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.124 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.141 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.192 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.197 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.210 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.219 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.228 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.240 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.249 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.255 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.261 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39.268 UTC [30347] WARNING: there is no transaction in progress
2026-08-06 13:07:39,848 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 13:07:39,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 13:07:40,092 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 13:07:40,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 13:07:40,329 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 13:07:40,329 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 13:07:40,591 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 13:07:40,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 13:07:40,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 13:07:41,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-q9h9.c.gcp-upstream.internal';"
2026-08-06 13:07:41,319 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 13:07:41,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 13:07:41,556 [I] DB_PATCHES have already been applied
I20260806 13:07:41.561805 14209 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXc7v3G/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 13:07:41.561869 14209 mini_ranger.cc:193] Using host: 127.13.224.84
I20260806 13:07:41.565649 14209 mini_ranger.cc:227] Ranger admin URL: http://127.13.224.84:54143
Aug 06, 2026 1:07:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:07:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:07:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 1:07:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 1:07:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 1:07:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 1:07:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 13:07:42.792145 14209 mini_ranger.cc:160] Time spent starting Ranger: real 8.377s user 0.000s sys 0.010s
Aug 06, 2026 1:08:02 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 1:08:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 1:08:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 1:08:02 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 1:08:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 1:08:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 1:08:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 1:08:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 1:08:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 1:08:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 1:08:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 1:08:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 1:08:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 1:08:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 1:08:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 1:08:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 1:08:02 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 1:08:03 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 13:08:04.214949 14209 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/rangerkms_127.13.224.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/rangerkms_127.13.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.224.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.224.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.224.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/HTTP_127.13.224.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.224.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/HTTP_127.13.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 13:08:04 dist-test-slave-q9h9 krb5kdc[30015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021684, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 13:08:04.288043 14209 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 13:08:04.298234 14209 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 13:08:05.027940 14209 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 13:08:05.028012 14209 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-kms
I20260806 13:08:05.028028 14209 mini_ranger_kms.cc:227] Using postgres at 127.13.224.84:34979
2026-08-06 13:08:05,527 [I] DB FLAVOR :POSTGRES
2026-08-06 13:08:05,528 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 13:08:05,528 [I] Checking connection
2026-08-06 13:08:05,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 13:08:05,812 [I] connection success
2026-08-06 13:08:05,813 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 13:08:05,813 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 13:08:05,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 13:08:06,044 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 13:08:06,044 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 13:08:06,045 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 13:08:06,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:34979/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 13:08:06,303 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 13:08:06.763968 14209 mini_ranger_kms.cc:310] Created kms service
I20260806 13:08:06.953473 14209 mini_ranger_kms.cc:326] Created kudu user
I20260806 13:08:06.997283 14209 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 13:08:06 dist-test-slave-q9h9 krb5kdc[30015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021684, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:08:07.349748 14209 mini_ranger_kms.cc:384] Added ranger policy
I20260806 13:08:07.349872 14209 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 13:08:07.349897 14209 mini_ranger_kms.cc:240] Using host: 127.13.224.84
I20260806 13:08:07.354079 14209 mini_ranger_kms.cc:276] Ranger KMS PID: 30634
I20260806 13:08:07.354208 14209 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.224.84:53681
Aug 06, 2026 1:08:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:08:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:08:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 1:08:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 1:08:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 1:08:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 13:08:09 dist-test-slave-q9h9 krb5kdc[30015](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786021689, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.224.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:09.777145 14209 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.489s user 0.000s sys 0.469s
I20260806 13:08:09.777251 14209 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 13:08:09.777333 14209 mini_ranger_kms.cc:401] 127.13.224.84:53681/kms/v1/keys
Aug 06 13:08:10 dist-test-slave-q9h9 krb5kdc[30015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021689, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.224.84@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
Aug 06 13:08:10 dist-test-slave-q9h9 krb5kdc[30015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021689, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.224.84@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
Aug 06 13:08:12 dist-test-slave-q9h9 krb5kdc[30015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:08:12.836987 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:46041
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:36231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:46041
--ranger_config_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.224.84:53681/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:08:12.936126 30709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:08:12.936311 30709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:08:12.936331 30709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:08:12.937440 30709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:08:12.937479 30709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:08:12.937491 30709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:08:12.937505 30709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:08:12.937527 30709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:08:12.939209 30709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:36231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.224.84:53681/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:46041
--ranger_config_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:46041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:08:12.939435 30709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:08:12.939638 30709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:08:12.944316 30721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:08:12.944373 30709 server_base.cc:1034] running on GCE node
W20260806 13:08:12.944355 30717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:12.944342 30715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:08:12.944766 30709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:08:12.945066 30709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:08:12.946229 30709 hybrid_clock.cc:648] HybridClock initialized: now 1786021692946210 us; error 35 us; skew 500 ppm
Aug 06 13:08:12 dist-test-slave-q9h9 krb5kdc[30015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021692, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:12.954926 30709 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:08:12.955358 30709 webserver.cc:460] Webserver started at http://127.13.224.126:40745/ using document root <none> and password file <none>
I20260806 13:08:12.955554 30709 fs_manager.cc:359] Metadata directory not provided
I20260806 13:08:12.955595 30709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:08:12.955719 30709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 13:08:12 dist-test-slave-q9h9 krb5kdc[30015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021692, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:08:13.012933 30709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "de0250d2ef5f45588a9f376a070a5d6b"
format_stamp: "Formatted at 2026-08-06 13:08:13 on dist-test-slave-q9h9"
server_key: "3c6e9a64943cc8adc241ac2756487c4b"
server_key_iv: "ac3b25f05bea1d5cfb6d3a808e55ca12"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:13.013422 30709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "de0250d2ef5f45588a9f376a070a5d6b"
format_stamp: "Formatted at 2026-08-06 13:08:13 on dist-test-slave-q9h9"
server_key: "3c6e9a64943cc8adc241ac2756487c4b"
server_key_iv: "ac3b25f05bea1d5cfb6d3a808e55ca12"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:13.015393 30709 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 13:08:13.029559 30733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:13.029790 30709 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:08:13.029891 30709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "de0250d2ef5f45588a9f376a070a5d6b"
format_stamp: "Formatted at 2026-08-06 13:08:13 on dist-test-slave-q9h9"
server_key: "3c6e9a64943cc8adc241ac2756487c4b"
server_key_iv: "ac3b25f05bea1d5cfb6d3a808e55ca12"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:13.030032 30709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:08:13.051911 30709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:08:13.053690 30709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:08:13.053809 30709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:08:13.061352 30709 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:46041
I20260806 13:08:13.061455 30812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:46041 every 8 connection(s)
I20260806 13:08:13.061729 30709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:08:13.062255 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 30709
I20260806 13:08:13.062350 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:08:13.062422 30813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:08:13.062527 14209 external_mini_cluster.cc:1428] Setting key 1644b04ebe16e287e86b860d7c625661
I20260806 13:08:13.065795 30813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b: Bootstrap starting.
I20260806 13:08:13.066711 30813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b: Neither blocks nor log segments found. Creating new log.
I20260806 13:08:13.067404 30813 log.cc:824] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b: Log is configured to *not* fsync() on all Append() calls
I20260806 13:08:13.068135 30813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b: No bootstrap required, opened a new log
Aug 06 13:08:13 dist-test-slave-q9h9 krb5kdc[30015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:08:13.070818 30813 raft_consensus.cc:348] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0250d2ef5f45588a9f376a070a5d6b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46041 } }
I20260806 13:08:13.070951 30813 raft_consensus.cc:374] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:08:13.070976 30813 raft_consensus.cc:729] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de0250d2ef5f45588a9f376a070a5d6b, State: Initialized, Role: FOLLOWER
I20260806 13:08:13.071099 30813 consensus_queue.cc:260] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0250d2ef5f45588a9f376a070a5d6b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46041 } }
I20260806 13:08:13.071193 30813 raft_consensus.cc:388] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:08:13.071236 30813 raft_consensus.cc:482] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:08:13.071285 30813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:08:13.071746 30813 raft_consensus.cc:504] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0250d2ef5f45588a9f376a070a5d6b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46041 } }
I20260806 13:08:13.071830 30813 leader_election.cc:302] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de0250d2ef5f45588a9f376a070a5d6b; no voters:
I20260806 13:08:13.072011 30813 leader_election.cc:288] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:08:13.072116 30824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:08:13.072336 30813 sys_catalog.cc:564] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:08:13.072369 30824 raft_consensus.cc:686] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [term 1 LEADER]: Becoming Leader. State: Replica: de0250d2ef5f45588a9f376a070a5d6b, State: Running, Role: LEADER
I20260806 13:08:13.072532 30824 consensus_queue.cc:237] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0250d2ef5f45588a9f376a070a5d6b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46041 } }
I20260806 13:08:13.072849 30824 sys_catalog.cc:455] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader de0250d2ef5f45588a9f376a070a5d6b. Latest consensus state: current_term: 1 leader_uuid: "de0250d2ef5f45588a9f376a070a5d6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0250d2ef5f45588a9f376a070a5d6b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46041 } } }
I20260806 13:08:13.073009 30824 sys_catalog.cc:458] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [sys.catalog]: This master's current role is: LEADER
I20260806 13:08:13.073354 30840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:08:13.073356 30821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:13.063016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56722 (local address 127.13.224.126:46041)
0806 13:08:13.063180 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:13.063188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:13.063241 (+ 53us) server_negotiation.cc:407] Connection header received
0806 13:08:13.063985 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:13.063993 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:13.064043 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:13.064159 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:08:13.064988 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:13.066193 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:13.067184 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:13.067614 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:13.067720 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:13.070162 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:08:13.070166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:08:13.070167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:08:13.070177 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:08:13.071807 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:13.072153 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:13.072155 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:13.072156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:13.072267 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:13.072467 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:13.072468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:13.072468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:13.072742 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:08:13.072939 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:13.073007 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 13:08:13.073117 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260806 13:08:13.073814 30825 sys_catalog.cc:455] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de0250d2ef5f45588a9f376a070a5d6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0250d2ef5f45588a9f376a070a5d6b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46041 } } }
I20260806 13:08:13.073966 30825 sys_catalog.cc:458] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b [sys.catalog]: This master's current role is: LEADER
I20260806 13:08:13.074393 30840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:08:13.075029 30813 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 13:08:13.080758 30840 catalog_manager.cc:1347] Generated new cluster ID: 398155b08a424231ab1090ebfb68d314
I20260806 13:08:13.080821 30840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:08:13.112028 30840 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:08:13.112914 30840 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:08:13.116040 30840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b: Generated new TSK 0
I20260806 13:08:13.116358 30840 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:08:13.953960 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.224.84:53681/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:46041
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:36231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:08:14.061967 30887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:08:14.062251 30887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:08:14.062282 30887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:08:14.063887 30887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:08:14.063941 30887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:08:14.063977 30887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:08:14.065541 30887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:36231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.224.84:53681/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:46041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:08:14.065745 30887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:08:14.065958 30887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:08:14.072656 30894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:14.072811 30892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:14.072705 30898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:08:14.073397 30887 server_base.cc:1034] running on GCE node
I20260806 13:08:14.073617 30887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:08:14.073844 30887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:08:14.075037 30887 hybrid_clock.cc:648] HybridClock initialized: now 1786021694075007 us; error 45 us; skew 500 ppm
Aug 06 13:08:14 dist-test-slave-q9h9 krb5kdc[30015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:14.077647 30887 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:08:14.078060 30887 webserver.cc:460] Webserver started at http://127.13.224.65:37379/ using document root <none> and password file <none>
I20260806 13:08:14.078262 30887 fs_manager.cc:359] Metadata directory not provided
I20260806 13:08:14.078308 30887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:08:14.078434 30887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 13:08:14 dist-test-slave-q9h9 krb5kdc[30015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:08:14.096558 30887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "bd0e474afedf4d3c8681d3749e067cac"
format_stamp: "Formatted at 2026-08-06 13:08:14 on dist-test-slave-q9h9"
server_key: "ec81dbc83f2d7e590770d2425fcb7dcf"
server_key_iv: "16759cc4c85748a0baaf586ce4202631"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:14.097580 30887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "bd0e474afedf4d3c8681d3749e067cac"
format_stamp: "Formatted at 2026-08-06 13:08:14 on dist-test-slave-q9h9"
server_key: "ec81dbc83f2d7e590770d2425fcb7dcf"
server_key_iv: "16759cc4c85748a0baaf586ce4202631"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:14.099504 30887 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 13:08:14.116305 30910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:14.116472 30887 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260806 13:08:14.116559 30887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "bd0e474afedf4d3c8681d3749e067cac"
format_stamp: "Formatted at 2026-08-06 13:08:14 on dist-test-slave-q9h9"
server_key: "ec81dbc83f2d7e590770d2425fcb7dcf"
server_key_iv: "16759cc4c85748a0baaf586ce4202631"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:14.116649 30887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:08:14.141990 30887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:08:14.144487 30887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:08:14.144779 30887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:08:14.145212 30887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:08:14.145644 30887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:08:14.145736 30887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:14.145846 30887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:08:14.145881 30887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:14.157612 30887 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:44083
I20260806 13:08:14.158911 30887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:08:14.159598 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 30887
I20260806 13:08:14.159703 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:08:14.159899 14209 external_mini_cluster.cc:1428] Setting key c6abf1e2150754732d5af86875e157e5
I20260806 13:08:14.160115 31102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:44083 every 8 connection(s)
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Aug 06 13:08:14 dist-test-slave-q9h9 krb5kdc[30015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
I20260806 13:08:14.175462 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.160975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:34503 (local address 127.13.224.126:46041)
0806 13:08:14.162269 (+ 1294us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:14.162271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:14.162281 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:08:14.162415 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:14.162416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:14.162429 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:14.162489 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:08:14.163159 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.163661 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.164954 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.165286 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.165373 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:14.171858 (+ 6485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:08:14.171859 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:08:14.171860 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:08:14.171868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:08:14.173758 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:14.174294 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:14.174295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:14.174296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:14.174365 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:14.174686 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:14.174687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:14.174687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:14.174842 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:08:14.174909 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:14.175254 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 13:08:14.175344 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1271,"thread_start_us":58,"threads_started":1}
I20260806 13:08:14.175664 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.160679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:46041 (local address 127.13.224.65:34503)
0806 13:08:14.161269 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:14.161315 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:14.162102 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:14.162611 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:14.162616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:14.162818 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:08:14.163061 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:14.163066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.163748 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.163749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:14.164851 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:14.164854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.165395 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.165396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:14.165542 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:14.170181 (+ 4639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:08:14.170192 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:08:14.171745 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:08:14.173853 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:14.173860 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:14.173862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:14.174207 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:14.174446 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:14.174447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:14.174448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:14.174601 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:14.174917 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:08:14.174925 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:08:14.175093 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 13:08:14.175213 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 13:08:14.175374 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":86,"threads_started":1}
I20260806 13:08:14.176177 31103 heartbeater.cc:347] Connected to a master server at 127.13.224.126:46041
I20260806 13:08:14.176288 31103 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:14.176487 31103 heartbeater.cc:511] Master 127.13.224.126:46041 requested a full tablet report, sending...
I20260806 13:08:14.177003 30755 ts_manager.cc:194] Registered new tserver with Master: bd0e474afedf4d3c8681d3749e067cac (127.13.224.65:44083)
I20260806 13:08:14.177606 30755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:34503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:08:14.192269 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.224.84:53681/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:46041
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:36231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:08:14.332481 31115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:08:14.332823 31115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:08:14.332878 31115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:08:14.334617 31115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:08:14.334672 31115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:08:14.334712 31115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:08:14.337071 31115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:36231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.224.84:53681/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:46041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:08:14.337337 31115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:08:14.337634 31115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:08:14.344765 31126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:14.344733 31122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:14.345026 31123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:08:14.346820 31115 server_base.cc:1034] running on GCE node
I20260806 13:08:14.347031 31115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:08:14.347262 31115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:08:14.348440 31115 hybrid_clock.cc:648] HybridClock initialized: now 1786021694348418 us; error 35 us; skew 500 ppm
Aug 06 13:08:14 dist-test-slave-q9h9 krb5kdc[30015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:14.351197 31115 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:08:14.351651 31115 webserver.cc:460] Webserver started at http://127.13.224.66:38799/ using document root <none> and password file <none>
I20260806 13:08:14.351955 31115 fs_manager.cc:359] Metadata directory not provided
I20260806 13:08:14.352053 31115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:08:14.352200 31115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 13:08:14 dist-test-slave-q9h9 krb5kdc[30015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
Aug 06 13:08:14 dist-test-slave-q9h9 krb5kdc[30015](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786021694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:14.364665 31115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "36b75c7200b34867a74d0f25c6ffd40f"
format_stamp: "Formatted at 2026-08-06 13:08:14 on dist-test-slave-q9h9"
server_key: "b4b66fda674ac2a0bdb36eba9fcf0118"
server_key_iv: "2a078cf4faea3a016f6bcd950fb553e4"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:14.365298 31115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "36b75c7200b34867a74d0f25c6ffd40f"
format_stamp: "Formatted at 2026-08-06 13:08:14 on dist-test-slave-q9h9"
server_key: "b4b66fda674ac2a0bdb36eba9fcf0118"
server_key_iv: "2a078cf4faea3a016f6bcd950fb553e4"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:14.368948 31115 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260806 13:08:14.399967 31137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:14.400254 31115 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260806 13:08:14.400353 31115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "36b75c7200b34867a74d0f25c6ffd40f"
format_stamp: "Formatted at 2026-08-06 13:08:14 on dist-test-slave-q9h9"
server_key: "b4b66fda674ac2a0bdb36eba9fcf0118"
server_key_iv: "2a078cf4faea3a016f6bcd950fb553e4"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:14.400450 31115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:08:14.427548 31115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:08:14.429522 31115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:08:14.429668 31115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:08:14.429939 31115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:08:14.430352 31115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:08:14.430393 31115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:14.430424 31115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:08:14.430447 31115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:14.445477 31115 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:43129
I20260806 13:08:14.446091 31115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:08:14.454468 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 31115
I20260806 13:08:14.454581 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:08:14.454749 14209 external_mini_cluster.cc:1428] Setting key 9e9c45f04d60e88a97994490b5e52b32
I20260806 13:08:14.461901 31318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:43129 every 8 connection(s)
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Aug 06 13:08:14 dist-test-slave-q9h9 krb5kdc[30015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:08:14.480108 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.462869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:54159 (local address 127.13.224.126:46041)
0806 13:08:14.462920 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:14.462922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:14.463148 (+ 226us) server_negotiation.cc:407] Connection header received
0806 13:08:14.463817 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:14.463818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:14.463835 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:14.463915 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:08:14.467607 (+ 3692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.468429 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.470473 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.470762 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.470814 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:14.477344 (+ 6530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:08:14.477346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:08:14.477347 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:08:14.477356 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:08:14.478822 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:14.479199 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:14.479200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:14.479201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:14.479254 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:14.479474 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:14.479475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:14.479476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:14.479588 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:08:14.479643 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:14.479918 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 13:08:14.480008 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 13:08:14.480239 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.462571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:46041 (local address 127.13.224.66:54159)
0806 13:08:14.463024 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:14.463072 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:14.463716 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:14.466822 (+ 3106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:14.466827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:14.467092 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:08:14.467468 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:14.467476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.468696 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.468697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:14.470363 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:14.470367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.470917 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.470918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:14.471105 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:14.471781 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:08:14.471792 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:08:14.477228 (+ 5436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:08:14.478899 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:14.478905 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:14.478908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:14.479121 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:14.479306 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:14.479307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:14.479308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:14.479412 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:14.479649 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:08:14.479655 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:08:14.479790 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 13:08:14.479872 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 13:08:14.480015 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":83,"threads_started":1}
I20260806 13:08:14.480657 31319 heartbeater.cc:347] Connected to a master server at 127.13.224.126:46041
I20260806 13:08:14.480751 31319 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:14.480922 31319 heartbeater.cc:511] Master 127.13.224.126:46041 requested a full tablet report, sending...
I20260806 13:08:14.481292 30755 ts_manager.cc:194] Registered new tserver with Master: 36b75c7200b34867a74d0f25c6ffd40f (127.13.224.66:43129)
I20260806 13:08:14.481794 30755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:54159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:08:14.492635 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.224.84:53681/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:46041
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:36231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:08:14.634341 31326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:08:14.634693 31326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:08:14.634752 31326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:08:14.636425 31326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:08:14.636483 31326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:08:14.636524 31326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:08:14.638904 31326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:36231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.224.84:53681/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:46041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:08:14.639163 31326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:08:14.639451 31326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:08:14.645807 31336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:14.646106 31338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:14.646140 31334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:08:14.646307 31326 server_base.cc:1034] running on GCE node
I20260806 13:08:14.646626 31326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:08:14.646873 31326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:08:14.654160 31326 hybrid_clock.cc:648] HybridClock initialized: now 1786021694654135 us; error 39 us; skew 500 ppm
Aug 06 13:08:14 dist-test-slave-q9h9 krb5kdc[30015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:14.656634 31326 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:08:14.657074 31326 webserver.cc:460] Webserver started at http://127.13.224.67:33847/ using document root <none> and password file <none>
I20260806 13:08:14.657308 31326 fs_manager.cc:359] Metadata directory not provided
I20260806 13:08:14.657367 31326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:08:14.657478 31326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 13:08:14 dist-test-slave-q9h9 krb5kdc[30015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:08:14.670264 31326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "375335b940d24917bbb3492bf6dfe3c6"
format_stamp: "Formatted at 2026-08-06 13:08:14 on dist-test-slave-q9h9"
server_key: "b19cd3c69557108ef67838665b8ea831"
server_key_iv: "e7224212dd447e3fc0063380f0145f72"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:14.670737 31326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "375335b940d24917bbb3492bf6dfe3c6"
format_stamp: "Formatted at 2026-08-06 13:08:14 on dist-test-slave-q9h9"
server_key: "b19cd3c69557108ef67838665b8ea831"
server_key_iv: "e7224212dd447e3fc0063380f0145f72"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:14.672444 31326 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 13:08:14.684652 31351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:14.685050 31326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:08:14.685148 31326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "375335b940d24917bbb3492bf6dfe3c6"
format_stamp: "Formatted at 2026-08-06 13:08:14 on dist-test-slave-q9h9"
server_key: "b19cd3c69557108ef67838665b8ea831"
server_key_iv: "e7224212dd447e3fc0063380f0145f72"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:14.685231 31326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:08:14.710278 31326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:08:14.711911 31326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:08:14.712014 31326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:08:14.712294 31326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:08:14.712615 31326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:08:14.712656 31326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:14.712682 31326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:08:14.712706 31326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:14.728366 31326 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:42425
I20260806 13:08:14.728912 31326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:08:14.729894 31516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:42425 every 8 connection(s)
I20260806 13:08:14.732316 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 31326
I20260806 13:08:14.732407 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:08:14.732575 14209 external_mini_cluster.cc:1428] Setting key 9bb6f9ecbf7d3aa4dc52124c71a4821b
Aug 06 13:08:14 dist-test-slave-q9h9 krb5kdc[30015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:08:14.740370 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.730508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:42659 (local address 127.13.224.126:46041)
0806 13:08:14.730560 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:14.730563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:14.730778 (+ 215us) server_negotiation.cc:407] Connection header received
0806 13:08:14.731288 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:14.731290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:14.731302 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:14.731380 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:08:14.731883 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.732629 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.733922 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.734248 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.734327 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:14.737185 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:08:14.737186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:08:14.737187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:08:14.737195 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:08:14.738839 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:14.739315 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:14.739316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:14.739317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:14.739377 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:14.739656 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:14.739657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:14.739657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:14.739801 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:08:14.739845 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:14.740200 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 13:08:14.740282 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 13:08:14.740857 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.730468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:46041 (local address 127.13.224.67:42659)
0806 13:08:14.730647 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:14.730688 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:14.731187 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:14.731432 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:14.731436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:14.731617 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:08:14.731795 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:14.731799 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.732718 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.732720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:14.733847 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:14.733850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.734329 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.734330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:14.734435 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:14.735372 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:08:14.735382 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:08:14.737056 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:08:14.738935 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:14.738942 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:14.738944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:14.739225 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:14.739451 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:14.739452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:14.739452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:14.739579 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:14.739891 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:08:14.739898 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:08:14.740046 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 13:08:14.740470 (+ 424us) client_negotiation.cc:240] Negotiation successful
0806 13:08:14.740626 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260806 13:08:14.741299 31517 heartbeater.cc:347] Connected to a master server at 127.13.224.126:46041
I20260806 13:08:14.741400 31517 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:14.741598 31517 heartbeater.cc:511] Master 127.13.224.126:46041 requested a full tablet report, sending...
I20260806 13:08:14.742046 30755 ts_manager.cc:194] Registered new tserver with Master: 375335b940d24917bbb3492bf6dfe3c6 (127.13.224.67:42425)
I20260806 13:08:14.742506 30755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:42659
I20260806 13:08:14.744241 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:08:14.753346 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.745913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56734 (local address 127.13.224.126:46041)
0806 13:08:14.745948 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:14.745950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:14.746158 (+ 208us) server_negotiation.cc:407] Connection header received
0806 13:08:14.746256 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:14.746257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:14.746267 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:14.746386 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:08:14.747165 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.747884 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.748939 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.749151 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.749221 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:14.750578 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:08:14.750580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:08:14.750580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:08:14.750587 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:08:14.752105 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:14.752491 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:14.752492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:14.752492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:14.752547 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:14.752770 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:14.752771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:14.752771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:14.752898 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:08:14.752972 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:14.753193 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 13:08:14.753264 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:08:14.757851 30755 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56734:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:08:14.760241 30755 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:08:14.770049 31533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.764162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36628 (local address 127.13.224.67:42425)
0806 13:08:14.764609 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:14.764615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:14.765371 (+ 756us) server_negotiation.cc:407] Connection header received
0806 13:08:14.765501 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:14.765504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:14.765521 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:14.765606 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:08:14.765865 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.767039 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.768739 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.769498 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.769546 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:14.769618 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:14.769895 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 13:08:14.769937 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":107,"threads_started":1}
I20260806 13:08:14.770370 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.763929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:42425 (local address 127.0.0.1:36628)
0806 13:08:14.765258 (+ 1329us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:14.765299 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:14.765371 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:14.765697 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:14.765701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:14.765704 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:08:14.765770 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:14.765772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.767244 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.767245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:14.768611 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:14.768614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.769684 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.769685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:14.769764 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:14.769778 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 13:08:14.770228 (+ 450us) client_negotiation.cc:240] Negotiation successful
0806 13:08:14.770294 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1295,"thread_start_us":961,"threads_started":1}
I20260806 13:08:14.771205 31422 tablet_service.cc:1467] Processing CreateTablet for tablet 59654ae3d2454dd29be1ffa3223c2444 (DEFAULT_TABLE table=test-table [id=3b16ea3464c14dbf94a8313100911285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:08:14.771589 31422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59654ae3d2454dd29be1ffa3223c2444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:08:14.772543 31534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.764114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57536 (local address 127.13.224.66:43129)
0806 13:08:14.764394 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:14.764399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:14.765594 (+ 1195us) server_negotiation.cc:407] Connection header received
0806 13:08:14.766153 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:14.766156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:14.766177 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:14.766238 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:08:14.766642 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.767773 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.771298 (+ 3525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.772104 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.772152 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:14.772218 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:14.772425 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 13:08:14.772464 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":150,"threads_started":1}
I20260806 13:08:14.772774 31536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.763943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:43129 (local address 127.0.0.1:57536)
0806 13:08:14.765532 (+ 1589us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:14.765541 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:14.765769 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:14.766510 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:14.766510 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:14.766512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:08:14.766544 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:14.766545 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.768024 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.768024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:14.769195 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:14.769198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.772280 (+ 3082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.772281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:14.772343 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:14.772346 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:08:14.772660 (+ 314us) client_negotiation.cc:240] Negotiation successful
0806 13:08:14.772703 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1566,"thread_start_us":1048,"threads_started":1}
I20260806 13:08:14.773483 31217 tablet_service.cc:1467] Processing CreateTablet for tablet 59654ae3d2454dd29be1ffa3223c2444 (DEFAULT_TABLE table=test-table [id=3b16ea3464c14dbf94a8313100911285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:08:14.773648 31541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.764444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:44083 (local address 127.0.0.1:50718)
0806 13:08:14.765660 (+ 1216us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:14.765669 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:14.765719 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:14.766324 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:14.766325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:14.766327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:08:14.766409 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:14.766411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.771176 (+ 4765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.771177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:14.772307 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:14.772310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.773403 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.773405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:14.773468 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:14.773472 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:08:14.773546 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 13:08:14.773580 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1200,"thread_start_us":110,"threads_started":1}
I20260806 13:08:14.773873 31538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.764770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50718 (local address 127.13.224.65:44083)
0806 13:08:14.764981 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:14.764985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:14.765821 (+ 836us) server_negotiation.cc:407] Connection header received
0806 13:08:14.765859 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:14.765862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:14.765878 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:14.765949 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:08:14.769297 (+ 3348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.771080 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.772410 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:14.773246 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:14.773318 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:14.773368 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:14.773750 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 13:08:14.773794 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":116,"threads_started":1}
I20260806 13:08:14.773928 31217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59654ae3d2454dd29be1ffa3223c2444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:08:14.774765 30995 tablet_service.cc:1467] Processing CreateTablet for tablet 59654ae3d2454dd29be1ffa3223c2444 (DEFAULT_TABLE table=test-table [id=3b16ea3464c14dbf94a8313100911285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:08:14.775101 30995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59654ae3d2454dd29be1ffa3223c2444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:08:14.778194 31546 tablet_bootstrap.cc:492] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6: Bootstrap starting.
I20260806 13:08:14.780082 31546 tablet_bootstrap.cc:654] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6: Neither blocks nor log segments found. Creating new log.
I20260806 13:08:14.780510 31546 log.cc:824] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6: Log is configured to *not* fsync() on all Append() calls
I20260806 13:08:14.782555 31549 tablet_bootstrap.cc:492] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f: Bootstrap starting.
I20260806 13:08:14.783654 31549 tablet_bootstrap.cc:654] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f: Neither blocks nor log segments found. Creating new log.
I20260806 13:08:14.784104 31549 log.cc:824] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f: Log is configured to *not* fsync() on all Append() calls
I20260806 13:08:14.784440 31552 tablet_bootstrap.cc:492] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac: Bootstrap starting.
I20260806 13:08:14.785475 31552 tablet_bootstrap.cc:654] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac: Neither blocks nor log segments found. Creating new log.
I20260806 13:08:14.786157 31552 log.cc:824] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac: Log is configured to *not* fsync() on all Append() calls
I20260806 13:08:14.786806 31549 tablet_bootstrap.cc:492] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f: No bootstrap required, opened a new log
I20260806 13:08:14.786861 31546 tablet_bootstrap.cc:492] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6: No bootstrap required, opened a new log
I20260806 13:08:14.786921 31549 ts_tablet_manager.cc:1397] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 13:08:14.789255 31546 ts_tablet_manager.cc:1397] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260806 13:08:14.790432 31549 raft_consensus.cc:348] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375335b940d24917bbb3492bf6dfe3c6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42425 } } peers { permanent_uuid: "36b75c7200b34867a74d0f25c6ffd40f" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43129 } } peers { permanent_uuid: "bd0e474afedf4d3c8681d3749e067cac" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44083 } }
I20260806 13:08:14.790712 31549 raft_consensus.cc:374] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:08:14.790804 31549 raft_consensus.cc:729] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36b75c7200b34867a74d0f25c6ffd40f, State: Initialized, Role: FOLLOWER
I20260806 13:08:14.791002 31549 consensus_queue.cc:260] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375335b940d24917bbb3492bf6dfe3c6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42425 } } peers { permanent_uuid: "36b75c7200b34867a74d0f25c6ffd40f" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43129 } } peers { permanent_uuid: "bd0e474afedf4d3c8681d3749e067cac" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44083 } }
I20260806 13:08:14.791421 31549 ts_tablet_manager.cc:1428] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 13:08:14.791602 31319 heartbeater.cc:503] Master 127.13.224.126:46041 was elected leader, sending a full tablet report...
I20260806 13:08:14.793247 31552 tablet_bootstrap.cc:492] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac: No bootstrap required, opened a new log
I20260806 13:08:14.793366 31552 ts_tablet_manager.cc:1397] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.003s
I20260806 13:08:14.793299 31546 raft_consensus.cc:348] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375335b940d24917bbb3492bf6dfe3c6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42425 } } peers { permanent_uuid: "36b75c7200b34867a74d0f25c6ffd40f" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43129 } } peers { permanent_uuid: "bd0e474afedf4d3c8681d3749e067cac" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44083 } }
I20260806 13:08:14.794135 31546 raft_consensus.cc:374] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:08:14.794225 31546 raft_consensus.cc:729] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375335b940d24917bbb3492bf6dfe3c6, State: Initialized, Role: FOLLOWER
I20260806 13:08:14.794406 31546 consensus_queue.cc:260] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375335b940d24917bbb3492bf6dfe3c6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42425 } } peers { permanent_uuid: "36b75c7200b34867a74d0f25c6ffd40f" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43129 } } peers { permanent_uuid: "bd0e474afedf4d3c8681d3749e067cac" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44083 } }
I20260806 13:08:14.795147 31546 ts_tablet_manager.cc:1428] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20260806 13:08:14.795310 31517 heartbeater.cc:503] Master 127.13.224.126:46041 was elected leader, sending a full tablet report...
I20260806 13:08:14.796792 31552 raft_consensus.cc:348] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375335b940d24917bbb3492bf6dfe3c6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42425 } } peers { permanent_uuid: "36b75c7200b34867a74d0f25c6ffd40f" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43129 } } peers { permanent_uuid: "bd0e474afedf4d3c8681d3749e067cac" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44083 } }
I20260806 13:08:14.796958 31552 raft_consensus.cc:374] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:08:14.796995 31552 raft_consensus.cc:729] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd0e474afedf4d3c8681d3749e067cac, State: Initialized, Role: FOLLOWER
I20260806 13:08:14.797107 31552 consensus_queue.cc:260] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375335b940d24917bbb3492bf6dfe3c6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42425 } } peers { permanent_uuid: "36b75c7200b34867a74d0f25c6ffd40f" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43129 } } peers { permanent_uuid: "bd0e474afedf4d3c8681d3749e067cac" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44083 } }
I20260806 13:08:14.797345 31552 ts_tablet_manager.cc:1428] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 13:08:14.797465 31103 heartbeater.cc:503] Master 127.13.224.126:46041 was elected leader, sending a full tablet report...
Aug 06 13:08:14 dist-test-slave-q9h9 krb5kdc[30015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
W20260806 13:08:14.911674 31105 tablet.cc:2369] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 13:08:14.963325 31322 tablet.cc:2369] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 13:08:14.979779 31518 tablet.cc:2369] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:08:14.998718 31556 raft_consensus.cc:482] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:08:14.998847 31556 raft_consensus.cc:504] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375335b940d24917bbb3492bf6dfe3c6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42425 } } peers { permanent_uuid: "36b75c7200b34867a74d0f25c6ffd40f" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43129 } } peers { permanent_uuid: "bd0e474afedf4d3c8681d3749e067cac" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44083 } }
I20260806 13:08:14.999253 31556 leader_election.cc:288] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 375335b940d24917bbb3492bf6dfe3c6 (127.13.224.67:42425), bd0e474afedf4d3c8681d3749e067cac (127.13.224.65:44083)
I20260806 13:08:15.003199 31533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.999581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:47955 (local address 127.13.224.67:42425)
0806 13:08:14.999627 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:14.999629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:15.000008 (+ 379us) server_negotiation.cc:407] Connection header received
0806 13:08:15.000109 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:15.000110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:15.000123 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:15.000151 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:08:15.000419 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.001174 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.002309 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.002815 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.002876 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:15.002885 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:15.003071 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 13:08:15.003124 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:08:15.003473 31560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.999518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:42425 (local address 127.13.224.66:47955)
0806 13:08:14.999947 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:14.999957 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:15.000038 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:15.000219 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:15.000220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:15.000222 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:08:15.000331 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:15.000333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.001305 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.001306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:15.002226 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:15.002229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.002915 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.002916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:15.002987 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:15.002992 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:08:15.003351 (+ 359us) client_negotiation.cc:240] Negotiation successful
0806 13:08:15.003397 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":100,"threads_started":1}
I20260806 13:08:15.003868 31436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59654ae3d2454dd29be1ffa3223c2444" candidate_uuid: "36b75c7200b34867a74d0f25c6ffd40f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "375335b940d24917bbb3492bf6dfe3c6" is_pre_election: true
I20260806 13:08:15.004027 31436 raft_consensus.cc:2454] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36b75c7200b34867a74d0f25c6ffd40f in term 0.
I20260806 13:08:15.004289 31138 leader_election.cc:302] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36b75c7200b34867a74d0f25c6ffd40f, 375335b940d24917bbb3492bf6dfe3c6; no voters:
I20260806 13:08:15.004446 31556 raft_consensus.cc:2781] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:08:15.004503 31556 raft_consensus.cc:482] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:08:15.004526 31556 raft_consensus.cc:3037] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:08:15.005154 31558 raft_consensus.cc:482] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:08:15.005265 31558 raft_consensus.cc:504] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375335b940d24917bbb3492bf6dfe3c6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42425 } } peers { permanent_uuid: "36b75c7200b34867a74d0f25c6ffd40f" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43129 } } peers { permanent_uuid: "bd0e474afedf4d3c8681d3749e067cac" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44083 } }
I20260806 13:08:15.005450 31556 raft_consensus.cc:504] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375335b940d24917bbb3492bf6dfe3c6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42425 } } peers { permanent_uuid: "36b75c7200b34867a74d0f25c6ffd40f" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43129 } } peers { permanent_uuid: "bd0e474afedf4d3c8681d3749e067cac" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44083 } }
I20260806 13:08:15.005589 31556 leader_election.cc:288] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [CANDIDATE]: Term 1 election: Requested vote from peers 375335b940d24917bbb3492bf6dfe3c6 (127.13.224.67:42425), bd0e474afedf4d3c8681d3749e067cac (127.13.224.65:44083)
I20260806 13:08:15.005571 31558 leader_election.cc:288] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 375335b940d24917bbb3492bf6dfe3c6 (127.13.224.67:42425), 36b75c7200b34867a74d0f25c6ffd40f (127.13.224.66:43129)
I20260806 13:08:15.005847 31436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59654ae3d2454dd29be1ffa3223c2444" candidate_uuid: "36b75c7200b34867a74d0f25c6ffd40f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "375335b940d24917bbb3492bf6dfe3c6"
I20260806 13:08:15.005916 31436 raft_consensus.cc:3037] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:08:15.006045 31538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.999862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:48877 (local address 127.13.224.65:44083)
0806 13:08:14.999902 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:14.999904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:15.002401 (+ 2497us) server_negotiation.cc:407] Connection header received
0806 13:08:15.002494 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:15.002495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:15.002506 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:15.002532 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:08:15.002856 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.003525 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.005010 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.005584 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.005665 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:15.005674 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:15.005907 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 13:08:15.005954 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:08:15.006258 31559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:14.999727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:44083 (local address 127.13.224.66:48877)
0806 13:08:15.002345 (+ 2618us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:15.002355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:15.002427 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:15.002596 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:15.002597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:15.002598 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:08:15.002768 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:15.002770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.003630 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.003630 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:15.004929 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:15.004932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.005722 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.005724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:15.005820 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:15.005826 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 13:08:15.006123 (+ 297us) client_negotiation.cc:240] Negotiation successful
0806 13:08:15.006167 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2602,"thread_start_us":49,"threads_started":1}
I20260806 13:08:15.006752 31436 raft_consensus.cc:2454] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36b75c7200b34867a74d0f25c6ffd40f in term 1.
I20260806 13:08:15.007025 31138 leader_election.cc:302] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36b75c7200b34867a74d0f25c6ffd40f, 375335b940d24917bbb3492bf6dfe3c6; no voters:
I20260806 13:08:15.007099 31027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59654ae3d2454dd29be1ffa3223c2444" candidate_uuid: "36b75c7200b34867a74d0f25c6ffd40f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd0e474afedf4d3c8681d3749e067cac" is_pre_election: true
I20260806 13:08:15.007155 31556 raft_consensus.cc:2781] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:08:15.007200 31027 raft_consensus.cc:2454] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36b75c7200b34867a74d0f25c6ffd40f in term 0.
I20260806 13:08:15.007295 31556 raft_consensus.cc:686] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [term 1 LEADER]: Becoming Leader. State: Replica: 36b75c7200b34867a74d0f25c6ffd40f, State: Running, Role: LEADER
I20260806 13:08:15.007303 31026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59654ae3d2454dd29be1ffa3223c2444" candidate_uuid: "36b75c7200b34867a74d0f25c6ffd40f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd0e474afedf4d3c8681d3749e067cac"
I20260806 13:08:15.007359 31026 raft_consensus.cc:3037] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:08:15.007380 31556 consensus_queue.cc:237] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375335b940d24917bbb3492bf6dfe3c6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42425 } } peers { permanent_uuid: "36b75c7200b34867a74d0f25c6ffd40f" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43129 } } peers { permanent_uuid: "bd0e474afedf4d3c8681d3749e067cac" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44083 } }
I20260806 13:08:15.008190 30754 catalog_manager.cc:5495] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f reported cstate change: term changed from 0 to 1, leader changed from <none> to 36b75c7200b34867a74d0f25c6ffd40f (127.13.224.66). New cstate: current_term: 1 leader_uuid: "36b75c7200b34867a74d0f25c6ffd40f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "375335b940d24917bbb3492bf6dfe3c6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36b75c7200b34867a74d0f25c6ffd40f" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd0e474afedf4d3c8681d3749e067cac" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44083 } health_report { overall_health: UNKNOWN } } }
I20260806 13:08:15.008360 31026 raft_consensus.cc:2454] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36b75c7200b34867a74d0f25c6ffd40f in term 1.
I20260806 13:08:15.012192 31533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:15.005791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:50637 (local address 127.13.224.67:42425)
0806 13:08:15.007133 (+ 1342us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:15.007135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:15.008859 (+ 1724us) server_negotiation.cc:407] Connection header received
0806 13:08:15.008947 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:15.008948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:15.008957 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:15.009009 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:08:15.009217 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.009906 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.011324 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.011826 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.011872 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:15.011879 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:15.012095 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 13:08:15.012134 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1320}
I20260806 13:08:15.012384 31561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:15.005756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:42425 (local address 127.13.224.65:50637)
0806 13:08:15.008650 (+ 2894us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:15.008660 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:15.008887 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:15.009052 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:15.009053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:15.009056 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:08:15.009139 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:15.009140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.009998 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.009998 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:15.011229 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:15.011232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.011935 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.011936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:15.012008 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:15.012011 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:08:15.012265 (+ 254us) client_negotiation.cc:240] Negotiation successful
0806 13:08:15.012306 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2876,"thread_start_us":142,"threads_started":1}
I20260806 13:08:15.012429 31534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:15.006306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:36777 (local address 127.13.224.66:43129)
0806 13:08:15.006341 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:15.006343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:15.009112 (+ 2769us) server_negotiation.cc:407] Connection header received
0806 13:08:15.009193 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:15.009194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:15.009202 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:15.009225 (+ 23us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:08:15.009449 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.010153 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.011473 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.011986 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.012032 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:15.012038 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:15.012332 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 13:08:15.012370 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:08:15.012650 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:15.006096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:43129 (local address 127.13.224.65:36777)
0806 13:08:15.009050 (+ 2954us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:15.009058 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:15.009135 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:15.009321 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:15.009322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:15.009323 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:08:15.009391 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:15.009392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.010245 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.010246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:15.011374 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:15.011377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.012109 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.012110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:15.012197 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:15.012200 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:08:15.012520 (+ 320us) client_negotiation.cc:240] Negotiation successful
0806 13:08:15.012559 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2937,"thread_start_us":133,"threads_started":1}
I20260806 13:08:15.012773 31436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59654ae3d2454dd29be1ffa3223c2444" candidate_uuid: "bd0e474afedf4d3c8681d3749e067cac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "375335b940d24917bbb3492bf6dfe3c6" is_pre_election: true
I20260806 13:08:15.012871 31436 raft_consensus.cc:2379] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd0e474afedf4d3c8681d3749e067cac in current term 1: Already voted for candidate 36b75c7200b34867a74d0f25c6ffd40f in this term.
I20260806 13:08:15.013329 31256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59654ae3d2454dd29be1ffa3223c2444" candidate_uuid: "bd0e474afedf4d3c8681d3749e067cac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36b75c7200b34867a74d0f25c6ffd40f" is_pre_election: true
I20260806 13:08:15.013607 30913 leader_election.cc:302] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd0e474afedf4d3c8681d3749e067cac; no voters: 36b75c7200b34867a74d0f25c6ffd40f, 375335b940d24917bbb3492bf6dfe3c6
I20260806 13:08:15.013726 31558 raft_consensus.cc:2726] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 13:08:15.059566 31534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:15.056239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.13.224.66:43129)
0806 13:08:15.056279 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:15.056282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:15.056291 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:08:15.056358 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:15.056359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:15.056383 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:15.056456 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:08:15.056683 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.057441 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.058538 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:15.058829 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:15.058899 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:15.059080 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:08:15.059213 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:08:15.059291 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:15.059346 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 13:08:15.059368 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:08:15.061993 31436 raft_consensus.cc:1261] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6 [term 1 FOLLOWER]: Refusing update from remote peer 36b75c7200b34867a74d0f25c6ffd40f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:08:15.062034 31026 raft_consensus.cc:1261] T 59654ae3d2454dd29be1ffa3223c2444 P bd0e474afedf4d3c8681d3749e067cac [term 1 FOLLOWER]: Refusing update from remote peer 36b75c7200b34867a74d0f25c6ffd40f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:08:15.062314 31556 consensus_queue.cc:1035] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [LEADER]: Connected to new peer: Peer: permanent_uuid: "375335b940d24917bbb3492bf6dfe3c6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:08:15.062470 31556 consensus_queue.cc:1035] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd0e474afedf4d3c8681d3749e067cac" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:08:15.067250 31573 mvcc.cc:205] Tried to move back new op lower bound from 7315544862970720256 to 7315544862750797824. Current Snapshot: MvccSnapshot[applied={T|T < 7315544862970720256}]
I20260806 13:08:15.067811 31575 mvcc.cc:205] Tried to move back new op lower bound from 7315544862970720256 to 7315544862750797824. Current Snapshot: MvccSnapshot[applied={T|T < 7315544862970720256}]
I20260806 13:08:15.068926 31574 mvcc.cc:205] Tried to move back new op lower bound from 7315544862970720256 to 7315544862750797824. Current Snapshot: MvccSnapshot[applied={T|T < 7315544862970720256}]
I20260806 13:08:15.071489 30754 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:08:15.071583 30754 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56734:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:08:15.072290 30754 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de0250d2ef5f45588a9f376a070a5d6b: Sending DeleteTablet for 3 replicas of tablet 59654ae3d2454dd29be1ffa3223c2444
I20260806 13:08:15.072842 31422 tablet_service.cc:1514] Processing DeleteTablet for tablet 59654ae3d2454dd29be1ffa3223c2444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:08:15 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:36628
I20260806 13:08:15.073203 31217 tablet_service.cc:1514] Processing DeleteTablet for tablet 59654ae3d2454dd29be1ffa3223c2444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:08:15 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:57536
I20260806 13:08:15.073625 30995 tablet_service.cc:1514] Processing DeleteTablet for tablet 59654ae3d2454dd29be1ffa3223c2444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:08:15 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:50718
I20260806 13:08:15.074229 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 30887
I20260806 13:08:15.087517 31591 tablet_replica.cc:330] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f: stopping tablet replica
I20260806 13:08:15.087631 31591 raft_consensus.cc:2229] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:08:15.087742 31591 raft_consensus.cc:2258] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:08:15.089335 31591 ts_tablet_manager.cc:1905] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:08:15.091112 31591 ts_tablet_manager.cc:1918] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:08:15.091171 31591 log.cc:1196] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/59654ae3d2454dd29be1ffa3223c2444
I20260806 13:08:15.091383 31591 ts_tablet_manager.cc:1939] T 59654ae3d2454dd29be1ffa3223c2444 P 36b75c7200b34867a74d0f25c6ffd40f: Deleting consensus metadata
I20260806 13:08:15.091742 30736 catalog_manager.cc:4841] TS 36b75c7200b34867a74d0f25c6ffd40f (127.13.224.66:43129): tablet 59654ae3d2454dd29be1ffa3223c2444 (table test-table [id=3b16ea3464c14dbf94a8313100911285]) successfully deleted
I20260806 13:08:15.110461 31588 tablet_replica.cc:330] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6: stopping tablet replica
I20260806 13:08:15.110692 31588 raft_consensus.cc:2229] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:08:15.110875 31588 raft_consensus.cc:2258] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 13:08:15.111367 30735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:44083 (error 108)
I20260806 13:08:15.111459 31588 ts_tablet_manager.cc:1905] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:08:15.112109 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 31115
I20260806 13:08:15.115000 31588 ts_tablet_manager.cc:1918] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:08:15.115087 31588 log.cc:1196] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/59654ae3d2454dd29be1ffa3223c2444
I20260806 13:08:15.115702 31588 ts_tablet_manager.cc:1939] T 59654ae3d2454dd29be1ffa3223c2444 P 375335b940d24917bbb3492bf6dfe3c6: Deleting consensus metadata
I20260806 13:08:15.116194 30734 catalog_manager.cc:4841] TS 375335b940d24917bbb3492bf6dfe3c6 (127.13.224.67:42425): tablet 59654ae3d2454dd29be1ffa3223c2444 (table test-table [id=3b16ea3464c14dbf94a8313100911285]) successfully deleted
I20260806 13:08:15.118347 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 31326
I20260806 13:08:15.120100 31536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:15.119976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:44083 (local address 127.0.0.1:50730)
0806 13:08:15.120026 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:15.120065 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:44083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 13:08:15.120223 30735 catalog_manager.cc:4584] TS bd0e474afedf4d3c8681d3749e067cac (127.13.224.65:44083): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 59654ae3d2454dd29be1ffa3223c2444: Network error: Client connection negotiation failed: client connection to 127.13.224.65:44083: connect: Connection refused (error 111)
I20260806 13:08:15.125317 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 30709
I20260806 13:08:15.139221 14209 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 13:08:15.482795 14209 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 13:08:15.482909 14209 mini_ranger.cc:66] Stopping Ranger...
I20260806 13:08:15.826408 14209 mini_ranger.cc:68] Stopped Ranger
2026-08-06 13:08:15.826 UTC [30062] LOG: received smart shutdown request
2026-08-06 13:08:15.830 UTC [30062] LOG: background worker "logical replication launcher" (PID 30076) exited with exit code 1
2026-08-06 13:08:15.830 UTC [30068] LOG: shutting down
2026-08-06 13:08:15.953 UTC [30062] LOG: database system is shut down
2026-08-06T13:08:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46196 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:08:16 dist-test-slave-q9h9 krb5kdc[31610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:08:16 dist-test-slave-q9h9 krb5kdc[31610](info): set up 2 sockets
Aug 06 13:08:16 dist-test-slave-q9h9 krb5kdc[31610](info): commencing operation
krb5kdc: starting...
W20260806 13:08:17.140377 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:08:17 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:08:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:08:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/rangeradmin_127.13.224.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/rangeradmin_127.13.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/rangerlookup_127.13.224.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/rangerlookup_127.13.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/HTTP_127.13.224.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/HTTP_127.13.224.84@KRBTEST.COM.keytab.
I20260806 13:08:17.283842 14209 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXc7v3G/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/postgres -l logfile start
2026-08-06 13:08:19.388 UTC [31662] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 13:08:19.388 UTC [31662] LOG: listening on IPv4 address "127.13.224.84", port 54317
2026-08-06 13:08:19.391 UTC [31662] LOG: listening on Unix socket "/tmp/.s.PGSQL.54317"
2026-08-06 13:08:19.409 UTC [31666] LOG: database system was shut down at 2026-08-06 13:08:18 UTC
2026-08-06 13:08:19.413 UTC [31662] LOG: database system is ready to accept connections
I20260806 13:08:20.436121 14209 mini_postgres.cc:96] Postgres bound to 54317
2026-08-06 13:08:20.443 UTC [31682] FATAL: database "slave" does not exist
127.13.224.84:54317 - accepting connections
I20260806 13:08:20.443442 14209 mini_ranger.cc:161] Starting Ranger...
I20260806 13:08:20.457983 14209 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 13:08:20.818879 14209 mini_ranger.cc:87] Created ranger Postgres database
I20260806 13:08:20.818964 14209 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-admin
2026-08-06 13:08:21,074 [I] DB FLAVOR :POSTGRES
2026-08-06 13:08:21,074 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 13:08:21,074 [I] Checking connection..
2026-08-06 13:08:21,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 13:08:21,328 [I] Checking connection passed.
2026-08-06 13:08:21,328 [I] --------- Verifying version history table ---------
2026-08-06 13:08:21,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 13:08:21,583 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 13:08:21,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 13:08:21,822 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 13:08:21,822 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 13:08:21,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 13:08:22.058 UTC [31786] WARNING: there is no transaction in progress
2026-08-06 13:08:22,065 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 13:08:22,065 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 13:08:22,288 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 13:08:22,288 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 13:08:22,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 13:08:22,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 13:08:22,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-q9h9.c.gcp-upstream.internal','N') ;"
2026-08-06 13:08:23,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 13:08:23,255 [I] Table x_portal_user does not exist in database ranger
2026-08-06 13:08:23,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 13:08:23,506 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 13:08:23,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 13:08:23,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 13:08:23,985 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 13:08:23,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 13:08:24.849 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:24.863 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:24.877 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:24.890 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:24.904 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:24.919 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:24.970 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:24.978 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:24.992 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:25.003 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:25.015 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:25.027 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:25.035 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:25.041 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:25.046 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:25.051 UTC [31954] WARNING: there is no transaction in progress
2026-08-06 13:08:25,577 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 13:08:25,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 13:08:25,809 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 13:08:25,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 13:08:26,069 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 13:08:26,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 13:08:26,326 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 13:08:26,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 13:08:26,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 13:08:26,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-q9h9.c.gcp-upstream.internal';"
2026-08-06 13:08:27,070 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 13:08:27,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 13:08:27,326 [I] DB_PATCHES have already been applied
I20260806 13:08:27.332002 14209 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXc7v3G/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 13:08:27.332062 14209 mini_ranger.cc:193] Using host: 127.13.224.84
I20260806 13:08:27.335745 14209 mini_ranger.cc:227] Ranger admin URL: http://127.13.224.84:43575
Aug 06, 2026 1:08:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:08:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:08:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 1:08:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 1:08:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 1:08:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 1:08:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 13:08:28.516230 14209 mini_ranger.cc:160] Time spent starting Ranger: real 8.073s user 0.006s sys 0.004s
Aug 06, 2026 1:08:47 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 1:08:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 1:08:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 1:08:47 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 1:08:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 1:08:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 1:08:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 1:08:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 1:08:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 1:08:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 1:08:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 1:08:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 1:08:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 1:08:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 1:08:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 1:08:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 1:08:47 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 1:08:48 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 13:08:49.255178 14209 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.224.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.224.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/rangerkms_127.13.224.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.224.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/rangerkms_127.13.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.224.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.224.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.224.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/HTTP_127.13.224.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.224.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/HTTP_127.13.224.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 13:08:49 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021729, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 13:08:49.327841 14209 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 13:08:49.336355 14209 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 13:08:49.942202 14209 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 13:08:49.942270 14209 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-kms
I20260806 13:08:49.942287 14209 mini_ranger_kms.cc:227] Using postgres at 127.13.224.84:54317
2026-08-06 13:08:50,107 [I] DB FLAVOR :POSTGRES
2026-08-06 13:08:50,107 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 13:08:50,108 [I] Checking connection
2026-08-06 13:08:50,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 13:08:50,361 [I] connection success
2026-08-06 13:08:50,361 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 13:08:50,361 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 13:08:50,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 13:08:50,602 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 13:08:50,602 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 13:08:50,602 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 13:08:50,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.224.84:54317/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 13:08:50,850 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 13:08:51.297542 14209 mini_ranger_kms.cc:310] Created kms service
I20260806 13:08:51.460225 14209 mini_ranger_kms.cc:326] Created kudu user
I20260806 13:08:51.501175 14209 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 13:08:51 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021729, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:08:51.865329 14209 mini_ranger_kms.cc:384] Added ranger policy
I20260806 13:08:51.865543 14209 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXc7v3G/build/release/bin/postgresql.jar:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXc7v3G/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 13:08:51.865569 14209 mini_ranger_kms.cc:240] Using host: 127.13.224.84
I20260806 13:08:51.870174 14209 mini_ranger_kms.cc:276] Ranger KMS PID: 32226
I20260806 13:08:51.870239 14209 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.224.84:36071
Aug 06, 2026 1:08:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:08:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:08:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 1:08:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 1:08:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 1:08:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 13:08:54 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786021734, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.224.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:54.308759 14209 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.981s user 0.012s sys 0.138s
I20260806 13:08:54.308863 14209 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 13:08:54.308907 14209 mini_ranger_kms.cc:401] 127.13.224.84:36071/kms/v1/keys
Aug 06 13:08:54 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021734, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.224.84@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
Aug 06 13:08:54 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021734, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.224.84@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
Aug 06 13:08:56 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:08:56.950263 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:37985
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:37985
--ranger_config_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.224.84:36071/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:08:57.041276 32288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:08:57.041450 32288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:08:57.041469 32288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:08:57.042587 32288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:08:57.042625 32288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:08:57.042637 32288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:08:57.042649 32288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:08:57.042660 32288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:08:57.044147 32288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.224.84:36071/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:37985
--ranger_config_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:37985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:08:57.044373 32288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:08:57.044589 32288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:08:57.049160 32295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:57.049194 32297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:57.049155 32299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:08:57.049428 32288 server_base.cc:1034] running on GCE node
I20260806 13:08:57.049641 32288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:08:57.049927 32288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:08:57.051075 32288 hybrid_clock.cc:648] HybridClock initialized: now 1786021737051062 us; error 30 us; skew 500 ppm
Aug 06 13:08:57 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021737, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:57.053409 32288 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:08:57.053815 32288 webserver.cc:460] Webserver started at http://127.13.224.126:34483/ using document root <none> and password file <none>
I20260806 13:08:57.054009 32288 fs_manager.cc:359] Metadata directory not provided
I20260806 13:08:57.054081 32288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:08:57.054204 32288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 13:08:57 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021737, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:08:57.113974 32288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "5892fd46bbdb4a40bf3b2af26dd5433a"
format_stamp: "Formatted at 2026-08-06 13:08:57 on dist-test-slave-q9h9"
server_key: "42f9f85b16838237e861ccecf1cffd94"
server_key_iv: "d6f7960c7e00f1b45b798708b4c5c49f"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:57.114387 32288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "5892fd46bbdb4a40bf3b2af26dd5433a"
format_stamp: "Formatted at 2026-08-06 13:08:57 on dist-test-slave-q9h9"
server_key: "42f9f85b16838237e861ccecf1cffd94"
server_key_iv: "d6f7960c7e00f1b45b798708b4c5c49f"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:57.115784 32288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:08:57.129976 32309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:57.130172 32288 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:08:57.130273 32288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "5892fd46bbdb4a40bf3b2af26dd5433a"
format_stamp: "Formatted at 2026-08-06 13:08:57 on dist-test-slave-q9h9"
server_key: "42f9f85b16838237e861ccecf1cffd94"
server_key_iv: "d6f7960c7e00f1b45b798708b4c5c49f"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:57.130347 32288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:08:57.155810 32288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:08:57.157533 32288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:08:57.157666 32288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:08:57.162202 32288 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:37985
I20260806 13:08:57.162232 32385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:37985 every 8 connection(s)
I20260806 13:08:57.162580 32288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:08:57.163219 32386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:08:57.165434 32386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a: Bootstrap starting.
I20260806 13:08:57.165827 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 32288
I20260806 13:08:57.165958 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:08:57.166070 32386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a: Neither blocks nor log segments found. Creating new log.
I20260806 13:08:57.166210 14209 external_mini_cluster.cc:1428] Setting key 68d3d2713ca9a81dc24be6c6dbe5d7be
I20260806 13:08:57.166369 32386 log.cc:824] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a: Log is configured to *not* fsync() on all Append() calls
I20260806 13:08:57.167138 32386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a: No bootstrap required, opened a new log
I20260806 13:08:57.169457 32386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37985 } }
I20260806 13:08:57.169617 32386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:08:57.169646 32386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5892fd46bbdb4a40bf3b2af26dd5433a, State: Initialized, Role: FOLLOWER
I20260806 13:08:57.169762 32386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37985 } }
I20260806 13:08:57.169821 32386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:08:57.169847 32386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:08:57.169885 32386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:08:57.170452 32386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37985 } }
I20260806 13:08:57.170549 32386 leader_election.cc:302] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5892fd46bbdb4a40bf3b2af26dd5433a; no voters:
I20260806 13:08:57.170737 32386 leader_election.cc:288] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:08:57.170770 32391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:08:57.170958 32391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 1 LEADER]: Becoming Leader. State: Replica: 5892fd46bbdb4a40bf3b2af26dd5433a, State: Running, Role: LEADER
I20260806 13:08:57.171099 32386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:08:57.171092 32391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37985 } }
I20260806 13:08:57.172055 32394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37985 } } }
I20260806 13:08:57.172135 32394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [sys.catalog]: This master's current role is: LEADER
I20260806 13:08:57.172508 32393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5892fd46bbdb4a40bf3b2af26dd5433a. Latest consensus state: current_term: 1 leader_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37985 } } }
I20260806 13:08:57.172546 32386 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 13:08:57.172566 32393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [sys.catalog]: This master's current role is: LEADER
Aug 06 13:08:57 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:08:57.177255 32408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:08:57.178632 32408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:08:57.180361 32390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:57.166742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.13.224.126:37985)
0806 13:08:57.167002 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:57.167011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:57.167039 (+ 28us) server_negotiation.cc:407] Connection header received
0806 13:08:57.167541 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:57.167551 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:57.167613 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:57.167753 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:08:57.168603 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:57.169937 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:57.171107 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:57.171757 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:57.171865 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:57.175124 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:08:57.175177 (+ 53us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:08:57.175178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:08:57.175290 (+ 112us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:08:57.178499 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:57.178996 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:57.178999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:57.178999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:57.179103 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:57.179335 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:57.179336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:57.179336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:57.179666 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:08:57.179848 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:57.179967 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 13:08:57.180132 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":76,"threads_started":1}
I20260806 13:08:57.187260 32408 catalog_manager.cc:1347] Generated new cluster ID: 1cea4440acc84387ab5c995708b9829e
I20260806 13:08:57.187309 32408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:08:57.197683 32408 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:08:57.198462 32408 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:08:57.207290 32408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a: Generated new TSK 0
I20260806 13:08:57.207506 32408 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:08:57.954190 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.224.84:36071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37985
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:08:58.071488 32446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:08:58.071849 32446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:08:58.071916 32446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:08:58.073625 32446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:08:58.073730 32446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:08:58.073809 32446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:08:58.075881 32446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.224.84:36071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:08:58.076125 32446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:08:58.076407 32446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:08:58.082549 32457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:58.082562 32453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:58.082553 32452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:08:58.083156 32446 server_base.cc:1034] running on GCE node
I20260806 13:08:58.083374 32446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:08:58.083630 32446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:08:58.084789 32446 hybrid_clock.cc:648] HybridClock initialized: now 1786021738084775 us; error 34 us; skew 500 ppm
Aug 06 13:08:58 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021738, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:58.087930 32446 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:08:58.088459 32446 webserver.cc:460] Webserver started at http://127.13.224.65:33273/ using document root <none> and password file <none>
I20260806 13:08:58.088742 32446 fs_manager.cc:359] Metadata directory not provided
I20260806 13:08:58.088805 32446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:08:58.088932 32446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 13:08:58 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021738, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:08:58.102957 32446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "3c6061244e894e78af9d55264728a186"
format_stamp: "Formatted at 2026-08-06 13:08:58 on dist-test-slave-q9h9"
server_key: "614025d2b39d7a36a6a5c5b74ba5488f"
server_key_iv: "ffa2fb3c65832e12a8241f676fed014d"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:58.103407 32446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "3c6061244e894e78af9d55264728a186"
format_stamp: "Formatted at 2026-08-06 13:08:58 on dist-test-slave-q9h9"
server_key: "614025d2b39d7a36a6a5c5b74ba5488f"
server_key_iv: "ffa2fb3c65832e12a8241f676fed014d"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:58.105098 32446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:08:58.117769 32467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:58.118068 32446 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:08:58.118184 32446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "3c6061244e894e78af9d55264728a186"
format_stamp: "Formatted at 2026-08-06 13:08:58 on dist-test-slave-q9h9"
server_key: "614025d2b39d7a36a6a5c5b74ba5488f"
server_key_iv: "ffa2fb3c65832e12a8241f676fed014d"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:58.118265 32446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:08:58.137774 32446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:08:58.139595 32446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:08:58.139724 32446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:08:58.139940 32446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:08:58.140278 32446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:08:58.140316 32446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:58.140343 32446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:08:58.140383 32446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:58.146647 32446 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:35365
I20260806 13:08:58.147460 32446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:08:58.146869 32632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:35365 every 8 connection(s)
I20260806 13:08:58.150794 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 32446
I20260806 13:08:58.150903 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:08:58.151096 14209 external_mini_cluster.cc:1428] Setting key 4b6a0ff899b7501c8c8fef9d618f62a5
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Aug 06 13:08:58 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021738, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:08:58.177327 32640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:58.154897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:48457 (local address 127.13.224.126:37985)
0806 13:08:58.155049 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:58.155051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:58.162257 (+ 7206us) server_negotiation.cc:407] Connection header received
0806 13:08:58.162949 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:58.162951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:58.162965 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:58.163014 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:08:58.163858 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:58.164664 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:58.166186 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:58.166488 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:58.166536 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:58.168968 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:08:58.168970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:08:58.168970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:08:58.168977 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:08:58.170654 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:58.176141 (+ 5487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:58.176142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:58.176143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:58.176214 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:58.176512 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:58.176513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:58.176513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:58.176673 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:08:58.176769 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:58.177124 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 13:08:58.177221 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260806 13:08:58.177531 32639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:58.154571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37985 (local address 127.13.224.65:48457)
0806 13:08:58.162122 (+ 7551us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:58.162171 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:58.162814 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:58.163228 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:58.163233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:58.163468 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:08:58.163746 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:58.163754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:58.164778 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:58.164779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:58.166074 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:58.166077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:58.166645 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:58.166647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:58.166783 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:58.167312 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:08:58.167323 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:08:58.168857 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:08:58.175698 (+ 6841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:58.175706 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:58.175708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:58.176015 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:58.176294 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:58.176295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:58.176296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:58.176441 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:58.176763 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:08:58.176770 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:08:58.176941 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 13:08:58.177048 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 13:08:58.177243 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7493,"thread_start_us":97,"threads_started":1}
I20260806 13:08:58.178110 32633 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37985
I20260806 13:08:58.178208 32633 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:58.178396 32633 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
I20260806 13:08:58.178933 32342 ts_manager.cc:194] Registered new tserver with Master: 3c6061244e894e78af9d55264728a186 (127.13.224.65:35365)
I20260806 13:08:58.179816 32342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:48457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:08:58.186259 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.224.84:36071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37985
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:08:58.320253 32642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:08:58.320483 32642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:08:58.320514 32642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:08:58.322108 32642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:08:58.322171 32642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:08:58.322209 32642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:08:58.324422 32642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.224.84:36071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:08:58.324921 32642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:08:58.325212 32642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:08:58.333243 32651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:58.333285 32655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
Aug 06 13:08:58 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786021738, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 13:08:58.333298 32652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:08:58.334456 32642 server_base.cc:1034] running on GCE node
I20260806 13:08:58.334663 32642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:08:58.334944 32642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:08:58.336115 32642 hybrid_clock.cc:648] HybridClock initialized: now 1786021738336093 us; error 36 us; skew 500 ppm
Aug 06 13:08:58 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021738, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:58.339468 32642 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:08:58.340015 32642 webserver.cc:460] Webserver started at http://127.13.224.66:35719/ using document root <none> and password file <none>
I20260806 13:08:58.340276 32642 fs_manager.cc:359] Metadata directory not provided
I20260806 13:08:58.340353 32642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:08:58.340553 32642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 13:08:58 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021738, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:08:58.357853 32642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "abf6d9751f844097a31c81cb12f7a8eb"
format_stamp: "Formatted at 2026-08-06 13:08:58 on dist-test-slave-q9h9"
server_key: "c4a1a0cdfe850624e16468f1de0957ac"
server_key_iv: "3d895987d11e970f7819282b4dc75397"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:58.358356 32642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "abf6d9751f844097a31c81cb12f7a8eb"
format_stamp: "Formatted at 2026-08-06 13:08:58 on dist-test-slave-q9h9"
server_key: "c4a1a0cdfe850624e16468f1de0957ac"
server_key_iv: "3d895987d11e970f7819282b4dc75397"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:58.362516 32642 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 13:08:58.378511 32666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:58.378772 32642 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 13:08:58.378868 32642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "abf6d9751f844097a31c81cb12f7a8eb"
format_stamp: "Formatted at 2026-08-06 13:08:58 on dist-test-slave-q9h9"
server_key: "c4a1a0cdfe850624e16468f1de0957ac"
server_key_iv: "3d895987d11e970f7819282b4dc75397"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:58.378947 32642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:08:58.390686 32642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:08:58.392974 32642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:08:58.393177 32642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:08:58.393524 32642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:08:58.394008 32642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:08:58.394133 32642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:58.394212 32642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:08:58.394275 32642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:58.407665 32642 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:37451
I20260806 13:08:58.408032 32642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:08:58.415125 361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:37451 every 8 connection(s)
I20260806 13:08:58.415693 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 32642
I20260806 13:08:58.415810 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:08:58.416023 14209 external_mini_cluster.cc:1428] Setting key ee8b8ae7d4af2c0ecb4e42dbf4237d86
Aug 06 13:08:58 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021738, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:08:58.431641 32640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:58.414968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:50495 (local address 127.13.224.126:37985)
0806 13:08:58.415033 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:58.415035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:58.416565 (+ 1530us) server_negotiation.cc:407] Connection header received
0806 13:08:58.419331 (+ 2766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:58.419333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:58.419351 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:58.419385 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:08:58.422060 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:58.422873 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:58.424522 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:58.424843 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:58.424890 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:58.428038 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:08:58.428040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:08:58.428040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:08:58.428049 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:08:58.429839 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:58.430401 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:58.430402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:58.430403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:58.430471 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:58.430826 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:58.430827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:58.430827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:58.430974 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:08:58.431036 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:58.431423 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 13:08:58.431521 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 13:08:58.432422 364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:58.414537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37985 (local address 127.13.224.66:50495)
0806 13:08:58.416391 (+ 1854us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:58.416443 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:58.419135 (+ 2692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:58.419596 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:58.419601 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:58.419902 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:08:58.420216 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:58.420224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:58.423042 (+ 2818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:58.423043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:58.424421 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:58.424424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:58.425027 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:58.425028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:58.425162 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:58.426164 (+ 1002us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:08:58.426176 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:08:58.427908 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:08:58.429951 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:58.429958 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:58.429961 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:58.430293 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:58.430572 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:58.430573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:58.430573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:58.430722 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:58.431077 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:08:58.431085 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:08:58.431244 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 13:08:58.432013 (+ 769us) client_negotiation.cc:240] Negotiation successful
0806 13:08:58.432190 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1769,"thread_start_us":188,"threads_started":1}
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
I20260806 13:08:58.433275 362 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37985
I20260806 13:08:58.433391 362 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:58.433601 362 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
I20260806 13:08:58.434046 32342 ts_manager.cc:194] Registered new tserver with Master: abf6d9751f844097a31c81cb12f7a8eb (127.13.224.66:37451)
I20260806 13:08:58.434506 32342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:50495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:08:58.452620 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.224.84:36071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37985
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:08:58.601631 370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:08:58.601955 370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:08:58.602056 370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:08:58.603771 370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:08:58.603961 370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:08:58.604045 370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:08:58.606492 370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.224.84:36071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:08:58.606984 370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:08:58.607270 370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:08:58.614699 383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:58.615291 382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:58.616070 385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:08:58.616349 370 server_base.cc:1034] running on GCE node
I20260806 13:08:58.616665 370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:08:58.616916 370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:08:58.618072 370 hybrid_clock.cc:648] HybridClock initialized: now 1786021738618049 us; error 36 us; skew 500 ppm
Aug 06 13:08:58 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021738, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:58.621598 370 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:08:58.622063 370 webserver.cc:460] Webserver started at http://127.13.224.67:35273/ using document root <none> and password file <none>
I20260806 13:08:58.622383 370 fs_manager.cc:359] Metadata directory not provided
I20260806 13:08:58.622536 370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:08:58.622705 370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 13:08:58 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021738, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:08:58.637478 370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "50f6ea8e718846c89f10834eacae7d3b"
format_stamp: "Formatted at 2026-08-06 13:08:58 on dist-test-slave-q9h9"
server_key: "a027ce4520d9ae01507216b1256bda01"
server_key_iv: "f1a8f8398b454c721590b173c6a4e198"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:58.637948 370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "50f6ea8e718846c89f10834eacae7d3b"
format_stamp: "Formatted at 2026-08-06 13:08:58 on dist-test-slave-q9h9"
server_key: "a027ce4520d9ae01507216b1256bda01"
server_key_iv: "f1a8f8398b454c721590b173c6a4e198"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:58.639969 370 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 13:08:58.655169 396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:58.655390 370 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 13:08:58.655473 370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "50f6ea8e718846c89f10834eacae7d3b"
format_stamp: "Formatted at 2026-08-06 13:08:58 on dist-test-slave-q9h9"
server_key: "a027ce4520d9ae01507216b1256bda01"
server_key_iv: "f1a8f8398b454c721590b173c6a4e198"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:58.655546 370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:08:58.689253 370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:08:58.693043 370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:08:58.693249 370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:08:58.693617 370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:08:58.694164 370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:08:58.694216 370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:58.694243 370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:08:58.694265 370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:58.709950 370 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:41793
I20260806 13:08:58.710003 559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:41793 every 8 connection(s)
I20260806 13:08:58.710491 370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:08:58.714254 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 370
I20260806 13:08:58.714376 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:08:58.714579 14209 external_mini_cluster.cc:1428] Setting key 8a0de46f0af3842b7a583c9b0f41f02b
Aug 06 13:08:58 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021738, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:08:58.724212 32640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:58.711405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:41061 (local address 127.13.224.126:37985)
0806 13:08:58.711452 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:58.711454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:58.711937 (+ 483us) server_negotiation.cc:407] Connection header received
0806 13:08:58.712766 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:58.712767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:58.712780 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:58.712842 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:08:58.713862 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:58.714665 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:58.717250 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:58.717587 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:58.717636 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:58.720844 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:08:58.720846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:08:58.720846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:08:58.720854 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:08:58.722521 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:58.723041 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:58.723042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:58.723044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:58.723110 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:58.723416 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:58.723417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:58.723417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:58.723563 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:08:58.723610 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:58.724024 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 13:08:58.724113 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:08:58.725086 562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:58.711447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37985 (local address 127.13.224.67:41061)
0806 13:08:58.711748 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:58.711811 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:58.712639 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:58.712933 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:58.712938 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:58.713340 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:08:58.713761 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:58.713769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:58.714861 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:58.714863 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:58.717106 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:58.717111 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:58.717804 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:58.717805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:58.717991 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:58.718662 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:08:58.718672 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:08:58.720724 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:08:58.722619 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:58.722627 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:58.722629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:58.722941 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:58.723185 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:58.723186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:58.723186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:58.723328 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:58.723662 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:08:58.723670 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:08:58.723853 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 13:08:58.724581 (+ 728us) client_negotiation.cc:240] Negotiation successful
0806 13:08:58.724772 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":121,"threads_started":1}
I20260806 13:08:58.726073 560 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37985
I20260806 13:08:58.726176 560 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:58.726370 560 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
I20260806 13:08:58.726727 32342 ts_manager.cc:194] Registered new tserver with Master: 50f6ea8e718846c89f10834eacae7d3b (127.13.224.67:41793)
I20260806 13:08:58.727192 32342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:41061
I20260806 13:08:58.748661 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:08:58.749552 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 32446
I20260806 13:08:58.779757 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 32642
I20260806 13:08:58.789590 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 370
I20260806 13:08:58.802006 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 32288
I20260806 13:08:58.812209 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:37985
--webserver_interface=127.13.224.126
--webserver_port=34483
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:37985
--ranger_config_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.224.84:36071/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.224.84:36071/kms with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:08:58.891803 565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:08:58.891986 565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:08:58.892014 565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:08:58.893095 565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:08:58.893137 565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:08:58.893157 565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:08:58.893172 565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:08:58.893188 565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:08:58.894827 565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.224.84:36071/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:37985
--ranger_config_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:37985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=34483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:08:58.895046 565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:08:58.895238 565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:08:58.900712 575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:58.900815 577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:58.900729 573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:08:58.900833 565 server_base.cc:1034] running on GCE node
I20260806 13:08:58.901198 565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:08:58.901501 565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:08:58.902678 565 hybrid_clock.cc:648] HybridClock initialized: now 1786021738902655 us; error 37 us; skew 500 ppm
Aug 06 13:08:58 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021738, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:58.905542 565 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:08:58.906055 565 webserver.cc:460] Webserver started at http://127.13.224.126:34483/ using document root <none> and password file <none>
I20260806 13:08:58.906306 565 fs_manager.cc:359] Metadata directory not provided
I20260806 13:08:58.906371 565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 13:08:58 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021738, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:08:58.948967 565 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:08:58.949676 586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:58.949839 565 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:08:58.949931 565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "5892fd46bbdb4a40bf3b2af26dd5433a"
format_stamp: "Formatted at 2026-08-06 13:08:57 on dist-test-slave-q9h9"
server_key: "42f9f85b16838237e861ccecf1cffd94"
server_key_iv: "d6f7960c7e00f1b45b798708b4c5c49f"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:58.950321 565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:08:58.963145 565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:08:58.964785 565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:08:58.964901 565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:08:58.969300 565 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:37985
I20260806 13:08:58.969328 657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:37985 every 8 connection(s)
I20260806 13:08:58.969681 565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:08:58.970883 659 sys_catalog.cc:263] Verifying existing consensus state
I20260806 13:08:58.971508 659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a: Bootstrap starting.
I20260806 13:08:58.978868 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 565
I20260806 13:08:58.979144 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.224.84:36071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:35365
--local_ip_for_outbound_sockets=127.13.224.65
--tserver_master_addrs=127.13.224.126:37985
--webserver_port=33273
--webserver_interface=127.13.224.65
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.224.84:36071/kms with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:08:58.979319 659 log.cc:824] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a: Log is configured to *not* fsync() on all Append() calls
I20260806 13:08:58.980877 659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:08:58.981107 659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a: Bootstrap complete.
I20260806 13:08:58.984067 659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37985 } }
I20260806 13:08:58.984383 659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5892fd46bbdb4a40bf3b2af26dd5433a, State: Initialized, Role: FOLLOWER
I20260806 13:08:58.984550 659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37985 } }
I20260806 13:08:58.984634 659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:08:58.984685 659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:08:58.984735 659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 1 FOLLOWER]: Advancing to term 2
I20260806 13:08:58.986042 659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37985 } }
I20260806 13:08:58.986158 659 leader_election.cc:302] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5892fd46bbdb4a40bf3b2af26dd5433a; no voters:
I20260806 13:08:58.986313 659 leader_election.cc:288] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 13:08:58.986356 665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 2 FOLLOWER]: Leader election won for term 2
I20260806 13:08:58.986522 665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [term 2 LEADER]: Becoming Leader. State: Replica: 5892fd46bbdb4a40bf3b2af26dd5433a, State: Running, Role: LEADER
I20260806 13:08:58.986562 659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:08:58.986641 665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37985 } }
I20260806 13:08:58.986886 665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5892fd46bbdb4a40bf3b2af26dd5433a. Latest consensus state: current_term: 2 leader_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37985 } } }
I20260806 13:08:58.986969 665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [sys.catalog]: This master's current role is: LEADER
I20260806 13:08:58.987010 667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5892fd46bbdb4a40bf3b2af26dd5433a" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37985 } } }
I20260806 13:08:58.987110 667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a [sys.catalog]: This master's current role is: LEADER
I20260806 13:08:58.987200 674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:08:58.987638 674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:08:58.987818 674 catalog_manager.cc:1259] Loaded cluster ID: 1cea4440acc84387ab5c995708b9829e
I20260806 13:08:58.987893 674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:08:58.988001 659 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 13:08:58.991881 674 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:08:58.992478 674 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a: Loaded TSK: 0
I20260806 13:08:58.992820 674 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 13:08:59.092453 664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:08:59.092794 664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:08:59.092854 664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:08:59.094516 664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:08:59.094571 664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:08:59.094621 664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:08:59.096727 664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.224.84:36071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:35365
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=33273
--tserver_master_addrs=127.13.224.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:08:59.096977 664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:08:59.097291 664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:08:59.103394 711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:59.103438 714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:59.103639 709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:08:59.104496 664 server_base.cc:1034] running on GCE node
I20260806 13:08:59.106231 664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:08:59.106492 664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:08:59.107647 664 hybrid_clock.cc:648] HybridClock initialized: now 1786021739107626 us; error 45 us; skew 500 ppm
Aug 06 13:08:59 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021739, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:59.110788 664 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:08:59.111266 664 webserver.cc:460] Webserver started at http://127.13.224.65:33273/ using document root <none> and password file <none>
I20260806 13:08:59.111505 664 fs_manager.cc:359] Metadata directory not provided
I20260806 13:08:59.111634 664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 13:08:59 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021739, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:08:59.129601 664 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:08:59.130676 727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:59.130837 664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 13:08:59.130923 664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "3c6061244e894e78af9d55264728a186"
format_stamp: "Formatted at 2026-08-06 13:08:58 on dist-test-slave-q9h9"
server_key: "614025d2b39d7a36a6a5c5b74ba5488f"
server_key_iv: "ffa2fb3c65832e12a8241f676fed014d"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:59.131381 664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:08:59.156625 664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:08:59.159062 664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:08:59.159230 664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:08:59.159487 664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:08:59.159857 664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:08:59.159906 664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:59.159937 664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:08:59.159994 664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:59.167508 664 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:35365
I20260806 13:08:59.168001 664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:08:59.177186 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 664
I20260806 13:08:59.177456 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.224.84:36071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:37451
--local_ip_for_outbound_sockets=127.13.224.66
--tserver_master_addrs=127.13.224.126:37985
--webserver_port=35719
--webserver_interface=127.13.224.66
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.224.84:36071/kms with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:08:59.178115 878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:35365 every 8 connection(s)
Aug 06 13:08:59 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021739, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:08:59.196223 883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:59.168959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37985 (local address 127.13.224.65:49039)
0806 13:08:59.169167 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:59.169218 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:59.169825 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:59.170542 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:59.170546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:59.170757 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:08:59.171014 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:59.171021 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.173894 (+ 2873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.173895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:59.180070 (+ 6175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:59.180073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.181072 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.181074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:59.181212 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:59.181741 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:08:59.181763 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:08:59.183364 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:08:59.193671 (+ 10307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:59.193679 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:59.193681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:59.193985 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:59.194848 (+ 863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:59.194849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:59.194849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:59.194990 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:59.195520 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:08:59.195528 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:08:59.195682 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 13:08:59.195797 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 13:08:59.195960 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260806 13:08:59.197480 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:59.169371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:49039 (local address 127.13.224.126:37985)
0806 13:08:59.169581 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:59.169590 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:59.169626 (+ 36us) server_negotiation.cc:407] Connection header received
0806 13:08:59.170190 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:59.170199 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:59.170264 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:59.170394 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:08:59.171162 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.173774 (+ 2612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.180161 (+ 6387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.180786 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.180897 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:59.183490 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:08:59.183495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:08:59.183495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:08:59.184883 (+ 1388us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:08:59.193575 (+ 8692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:59.194614 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:59.194617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:59.194618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:59.194731 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:59.195093 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:59.195094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:59.195094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:59.195406 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:08:59.195488 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:59.197154 (+ 1666us) server_negotiation.cc:299] Negotiation successful
0806 13:08:59.197280 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":80,"threads_started":1}
I20260806 13:08:59.198216 880 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37985
I20260806 13:08:59.198313 880 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:59.198518 880 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
W20260806 13:08:59.199388 880 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:37985 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 13:08:59.199456 880 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:59.199515 880 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
I20260806 13:08:59.199875 880 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:59.199944 880 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
W20260806 13:08:59.200227 880 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 13:08:59.283107 886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:08:59.283367 886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:08:59.283401 886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:08:59.284929 886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:08:59.284986 886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:08:59.285023 886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:08:59.286940 886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.224.84:36071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:37451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=35719
--tserver_master_addrs=127.13.224.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:08:59.287166 886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:08:59.287468 886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:08:59.292723 891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:59.293155 893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:59.293483 895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:08:59.293826 886 server_base.cc:1034] running on GCE node
I20260806 13:08:59.294076 886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:08:59.294353 886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:08:59.295512 886 hybrid_clock.cc:648] HybridClock initialized: now 1786021739295481 us; error 43 us; skew 500 ppm
Aug 06 13:08:59 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021739, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:59.298624 886 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:08:59.299026 886 webserver.cc:460] Webserver started at http://127.13.224.66:35719/ using document root <none> and password file <none>
I20260806 13:08:59.299255 886 fs_manager.cc:359] Metadata directory not provided
I20260806 13:08:59.299311 886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 13:08:59 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021739, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:08:59.315040 886 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:08:59.315670 910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:59.315825 886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:08:59.315915 886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "abf6d9751f844097a31c81cb12f7a8eb"
format_stamp: "Formatted at 2026-08-06 13:08:58 on dist-test-slave-q9h9"
server_key: "c4a1a0cdfe850624e16468f1de0957ac"
server_key_iv: "3d895987d11e970f7819282b4dc75397"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:59.316267 886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:08:59.331310 886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:08:59.332985 886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:08:59.333117 886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:08:59.333324 886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:08:59.333629 886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:08:59.333668 886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:59.333694 886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:08:59.333719 886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:59.340342 886 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:37451
I20260806 13:08:59.340428 1059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:37451 every 8 connection(s)
I20260806 13:08:59.340723 886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:08:59.345696 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 886
I20260806 13:08:59.345955 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.224.84:36071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:41793
--local_ip_for_outbound_sockets=127.13.224.67
--tserver_master_addrs=127.13.224.126:37985
--webserver_port=35273
--webserver_interface=127.13.224.67
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.224.84:36071/kms with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 13:08:59 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021739, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:08:59.352057 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:59.341445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37985 (local address 127.13.224.66:44141)
0806 13:08:59.341624 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:59.341673 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:59.342322 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:59.342563 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:59.342579 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:59.342832 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:08:59.343099 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:59.343106 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.343757 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.343759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:59.345028 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:59.345031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.345637 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.345638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:59.345783 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:59.346312 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:08:59.346322 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:08:59.348708 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:08:59.350426 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:59.350434 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:59.350436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:59.350741 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:59.350965 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:59.350966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:59.350967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:59.351109 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:59.351434 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:08:59.351441 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:08:59.351603 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 13:08:59.351704 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 13:08:59.351845 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260806 13:08:59.353309 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:59.341460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:44141 (local address 127.13.224.126:37985)
0806 13:08:59.341528 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:59.341530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:59.341771 (+ 241us) server_negotiation.cc:407] Connection header received
0806 13:08:59.342424 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:59.342426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:59.342439 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:59.342476 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:08:59.343183 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.343664 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.345142 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.345446 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.345499 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:59.348820 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:08:59.348821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:08:59.348821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:08:59.348829 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:08:59.350348 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:59.350826 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:59.350827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:59.350827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:59.350897 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:59.351201 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:59.351203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:59.351203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:59.351344 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:08:59.351406 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:59.353121 (+ 1715us) server_negotiation.cc:299] Negotiation successful
0806 13:08:59.353215 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 13:08:59.353674 1060 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37985
I20260806 13:08:59.353775 1060 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:59.353988 1060 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
W20260806 13:08:59.354682 1060 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:37985 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 13:08:59.354763 1060 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:59.354851 1060 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
I20260806 13:08:59.355161 1060 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:59.355238 1060 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
W20260806 13:08:59.355523 1060 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 13:08:59.446820 1063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:08:59.447009 1063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:08:59.447041 1063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:08:59.448161 1063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:08:59.448201 1063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:08:59.448235 1063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:08:59.449781 1063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.224.84:36071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:41793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=35273
--tserver_master_addrs=127.13.224.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:08:59.449997 1063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:08:59.450237 1063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:08:59.455008 1068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:59.455016 1071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:08:59.455312 1069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:08:59.457984 1063 server_base.cc:1034] running on GCE node
I20260806 13:08:59.458190 1063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:08:59.458420 1063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:08:59.459571 1063 hybrid_clock.cc:648] HybridClock initialized: now 1786021739459551 us; error 34 us; skew 500 ppm
Aug 06 13:08:59 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021739, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:08:59.461900 1063 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:08:59.462342 1063 webserver.cc:460] Webserver started at http://127.13.224.67:35273/ using document root <none> and password file <none>
I20260806 13:08:59.462549 1063 fs_manager.cc:359] Metadata directory not provided
I20260806 13:08:59.462610 1063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 13:08:59 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021739, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:08:59.480690 1063 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 13:08:59.486510 1085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:59.486863 1063 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260806 13:08:59.486979 1063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "50f6ea8e718846c89f10834eacae7d3b"
format_stamp: "Formatted at 2026-08-06 13:08:58 on dist-test-slave-q9h9"
server_key: "a027ce4520d9ae01507216b1256bda01"
server_key_iv: "f1a8f8398b454c721590b173c6a4e198"
server_key_version: "kuduclusterkey@0"
I20260806 13:08:59.487493 1063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:08:59.503003 1063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:08:59.504958 1063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:08:59.505120 1063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:08:59.505344 1063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:08:59.505692 1063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:08:59.505748 1063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:59.505784 1063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:08:59.505811 1063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:08:59.513872 1063 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:41793
I20260806 13:08:59.513958 1242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:41793 every 8 connection(s)
I20260806 13:08:59.514345 1063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:08:59.514618 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 1063
Aug 06 13:08:59 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021739, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:08:59.523432 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:59.515211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58198 (local address 127.13.224.126:37985)
0806 13:08:59.515723 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:59.515725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:59.515732 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:08:59.515759 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:59.515760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:59.515775 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:59.515847 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:08:59.516617 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.517387 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.518509 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.518795 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.518861 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:59.520344 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:08:59.520346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:08:59.520346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:08:59.520354 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:08:59.522082 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:59.522465 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:59.522467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:59.522467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:59.522534 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:59.522735 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:59.522736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:59.522736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:59.522875 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:08:59.523059 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:59.523100 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 13:08:59.523333 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":135,"threads_started":1}
I20260806 13:08:59.525151 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:59.515146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:58377 (local address 127.13.224.126:37985)
0806 13:08:59.515197 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 13:08:59.515198 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:08:59.515721 (+ 523us) server_negotiation.cc:407] Connection header received
0806 13:08:59.516279 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:08:59.516280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:08:59.516292 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:08:59.516353 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:08:59.516993 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.517652 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.518916 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.519156 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.519203 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:59.521738 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:08:59.521739 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:08:59.521739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:08:59.521747 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:08:59.523372 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:59.524178 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:59.524179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:59.524179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:59.524240 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:08:59.524472 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:08:59.524473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:08:59.524473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:08:59.524585 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:08:59.524645 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 13:08:59.524976 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 13:08:59.525058 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 13:08:59.525797 1247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:08:59.515195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37985 (local address 127.13.224.67:58377)
0806 13:08:59.515384 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 13:08:59.515638 (+ 254us) client_negotiation.cc:174] Beginning negotiation
0806 13:08:59.516180 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:08:59.516423 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:08:59.516427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:08:59.516621 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:08:59.516875 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:59.516882 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.517746 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.517747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:59.518741 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:08:59.518744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:08:59.519320 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:08:59.519321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:08:59.519455 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:08:59.519894 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:08:59.519903 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:08:59.521620 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:08:59.523453 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:59.523461 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:59.523463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:59.524081 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:59.524300 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:08:59.524300 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:08:59.524301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:08:59.524411 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:08:59.524646 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:08:59.524652 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:08:59.524800 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 13:08:59.525460 (+ 660us) client_negotiation.cc:240] Negotiation successful
0806 13:08:59.525639 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260806 13:08:59.526268 1244 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37985
I20260806 13:08:59.526360 1244 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:59.526535 1244 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
W20260806 13:08:59.527181 1244 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:37985 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 13:08:59.527247 1244 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:59.527319 1244 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
I20260806 13:08:59.527622 1244 heartbeater.cc:464] Registering TS with master...
I20260806 13:08:59.527700 1244 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
W20260806 13:08:59.527997 1244 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 13:09:00 dist-test-slave-q9h9 krb5kdc[31610](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786021740, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:00.200419 880 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:00.200598 880 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
I20260806 13:09:00.200949 608 ts_manager.cc:194] Registered new tserver with Master: 3c6061244e894e78af9d55264728a186 (127.13.224.65:35365)
I20260806 13:09:00.201989 608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:49039
I20260806 13:09:00.355705 1060 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:00.355863 1060 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
I20260806 13:09:00.356169 608 ts_manager.cc:194] Registered new tserver with Master: abf6d9751f844097a31c81cb12f7a8eb (127.13.224.66:37451)
I20260806 13:09:00.356712 608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:44141
Aug 06 13:09:00 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021740, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:09:00.528168 1244 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:00.528311 1244 heartbeater.cc:511] Master 127.13.224.126:37985 requested a full tablet report, sending...
I20260806 13:09:00.528599 608 ts_manager.cc:194] Registered new tserver with Master: 50f6ea8e718846c89f10834eacae7d3b (127.13.224.67:41793)
I20260806 13:09:00.528873 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:09:00.529135 608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:58377
I20260806 13:09:00.540429 1279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.529853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58204 (local address 127.13.224.126:37985)
0806 13:09:00.530195 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:00.530197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:00.530342 (+ 145us) server_negotiation.cc:407] Connection header received
0806 13:09:00.530436 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:00.530437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:00.530449 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:00.530489 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:00.531238 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.531997 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.533021 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.533339 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.533411 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:00.537356 (+ 3945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:00.537358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:00.537358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:00.537366 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:00.539187 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:00.539585 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:00.539586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:00.539587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:00.539659 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:00.539866 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:00.539867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:00.539867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:00.540006 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:00.540110 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:00.540240 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 13:09:00.540331 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":71,"threads_started":1}
I20260806 13:09:00.542636 608 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:09:00.544958 608 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:09:00.553459 1295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.549060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59296 (local address 127.13.224.66:37451)
0806 13:09:00.549201 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:00.549205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:00.549216 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:09:00.549251 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:00.549253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:00.549270 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:00.549521 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:00.549607 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.550670 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.552280 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.553040 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.553125 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:00.553190 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:00.553332 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 13:09:00.553376 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260806 13:09:00.553480 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.548163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:37451 (local address 127.0.0.1:59296)
0806 13:09:00.548729 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:00.548778 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:00.548844 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:00.549383 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:00.549387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:00.549389 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:00.549453 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:00.549455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.550752 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.550753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:00.552021 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:00.552024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.553128 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.553130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:00.553218 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:00.553233 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 13:09:00.553310 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 13:09:00.553360 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":107,"threads_started":1}
I20260806 13:09:00.554622 947 tablet_service.cc:1467] Processing CreateTablet for tablet 99c8ecfae533435e9f6f76e883414102 (DEFAULT_TABLE table=test-table [id=65d666a4e11b4a7ea2b278f6ccc62351]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:00.555065 947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99c8ecfae533435e9f6f76e883414102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:00.556841 1290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.548166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:41793 (local address 127.0.0.1:37870)
0806 13:09:00.549500 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:00.549511 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:00.549557 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:00.550351 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:00.550352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:00.550354 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:00.550418 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:00.550420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.554478 (+ 4058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.554479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:00.555565 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:00.555567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.556604 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.556605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:00.556693 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:00.556697 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:09:00.556740 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 13:09:00.556779 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1317,"thread_start_us":94,"threads_started":1}
I20260806 13:09:00.557049 1292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.548418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37870 (local address 127.13.224.67:41793)
0806 13:09:00.550212 (+ 1794us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:00.550216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:00.550226 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:09:00.550254 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:00.550255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:00.550268 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:00.550343 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:00.550527 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.554394 (+ 3867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.555650 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.556515 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.556590 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:00.556648 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:00.556764 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 13:09:00.556804 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1763,"thread_start_us":54,"threads_started":1}
I20260806 13:09:00.557940 1133 tablet_service.cc:1467] Processing CreateTablet for tablet 99c8ecfae533435e9f6f76e883414102 (DEFAULT_TABLE table=test-table [id=65d666a4e11b4a7ea2b278f6ccc62351]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:00.558296 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.548439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:35365 (local address 127.0.0.1:40598)
0806 13:09:00.549634 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:00.549643 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:00.549722 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:00.550007 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:00.550008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:00.550010 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:00.550087 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:00.550089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.551118 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.551119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:00.552183 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:00.552186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.558079 (+ 5893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.558080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:00.558151 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:00.558154 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:09:00.558202 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 13:09:00.558233 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1182,"thread_start_us":215,"threads_started":1}
I20260806 13:09:00.558373 1133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99c8ecfae533435e9f6f76e883414102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:00.560132 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.548837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40598 (local address 127.13.224.65:35365)
0806 13:09:00.549023 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:00.549028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:00.549694 (+ 666us) server_negotiation.cc:407] Connection header received
0806 13:09:00.549822 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:00.549825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:00.549843 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:00.549900 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:00.550186 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.551045 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.552279 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.553133 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.553191 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:00.553238 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:00.559986 (+ 6748us) server_negotiation.cc:299] Negotiation successful
0806 13:09:00.560030 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":118,"threads_started":1}
I20260806 13:09:00.560840 776 tablet_service.cc:1467] Processing CreateTablet for tablet 99c8ecfae533435e9f6f76e883414102 (DEFAULT_TABLE table=test-table [id=65d666a4e11b4a7ea2b278f6ccc62351]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:00.561172 776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99c8ecfae533435e9f6f76e883414102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:00.564610 1299 tablet_bootstrap.cc:492] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186: Bootstrap starting.
I20260806 13:09:00.565222 1298 tablet_bootstrap.cc:492] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b: Bootstrap starting.
I20260806 13:09:00.565744 1299 tablet_bootstrap.cc:654] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:00.566179 1298 tablet_bootstrap.cc:654] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:00.566483 1299 log.cc:824] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:00.566577 1298 log.cc:824] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:00.566956 1296 tablet_bootstrap.cc:492] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb: Bootstrap starting.
I20260806 13:09:00.567592 1298 tablet_bootstrap.cc:492] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b: No bootstrap required, opened a new log
I20260806 13:09:00.567710 1298 ts_tablet_manager.cc:1397] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 13:09:00.567859 1296 tablet_bootstrap.cc:654] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:00.568351 1296 log.cc:824] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:00.569020 1299 tablet_bootstrap.cc:492] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186: No bootstrap required, opened a new log
I20260806 13:09:00.569089 1299 ts_tablet_manager.cc:1397] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260806 13:09:00.571910 1298 raft_consensus.cc:348] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abf6d9751f844097a31c81cb12f7a8eb" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37451 } } peers { permanent_uuid: "3c6061244e894e78af9d55264728a186" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35365 } } peers { permanent_uuid: "50f6ea8e718846c89f10834eacae7d3b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41793 } }
I20260806 13:09:00.572196 1298 raft_consensus.cc:374] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:00.572300 1298 raft_consensus.cc:729] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50f6ea8e718846c89f10834eacae7d3b, State: Initialized, Role: FOLLOWER
I20260806 13:09:00.572521 1298 consensus_queue.cc:260] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abf6d9751f844097a31c81cb12f7a8eb" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37451 } } peers { permanent_uuid: "3c6061244e894e78af9d55264728a186" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35365 } } peers { permanent_uuid: "50f6ea8e718846c89f10834eacae7d3b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41793 } }
I20260806 13:09:00.572713 1299 raft_consensus.cc:348] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abf6d9751f844097a31c81cb12f7a8eb" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37451 } } peers { permanent_uuid: "3c6061244e894e78af9d55264728a186" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35365 } } peers { permanent_uuid: "50f6ea8e718846c89f10834eacae7d3b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41793 } }
I20260806 13:09:00.572880 1299 raft_consensus.cc:374] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:00.572927 1299 raft_consensus.cc:729] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c6061244e894e78af9d55264728a186, State: Initialized, Role: FOLLOWER
I20260806 13:09:00.573057 1298 ts_tablet_manager.cc:1428] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260806 13:09:00.573225 1244 heartbeater.cc:503] Master 127.13.224.126:37985 was elected leader, sending a full tablet report...
I20260806 13:09:00.573375 1299 consensus_queue.cc:260] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abf6d9751f844097a31c81cb12f7a8eb" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37451 } } peers { permanent_uuid: "3c6061244e894e78af9d55264728a186" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35365 } } peers { permanent_uuid: "50f6ea8e718846c89f10834eacae7d3b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41793 } }
I20260806 13:09:00.573637 1299 ts_tablet_manager.cc:1428] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260806 13:09:00.573761 880 heartbeater.cc:503] Master 127.13.224.126:37985 was elected leader, sending a full tablet report...
I20260806 13:09:00.574944 1296 tablet_bootstrap.cc:492] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb: No bootstrap required, opened a new log
I20260806 13:09:00.575018 1296 ts_tablet_manager.cc:1397] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260806 13:09:00.577503 1296 raft_consensus.cc:348] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abf6d9751f844097a31c81cb12f7a8eb" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37451 } } peers { permanent_uuid: "3c6061244e894e78af9d55264728a186" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35365 } } peers { permanent_uuid: "50f6ea8e718846c89f10834eacae7d3b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41793 } }
I20260806 13:09:00.577627 1296 raft_consensus.cc:374] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:00.577657 1296 raft_consensus.cc:729] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abf6d9751f844097a31c81cb12f7a8eb, State: Initialized, Role: FOLLOWER
I20260806 13:09:00.577881 1296 consensus_queue.cc:260] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abf6d9751f844097a31c81cb12f7a8eb" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37451 } } peers { permanent_uuid: "3c6061244e894e78af9d55264728a186" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35365 } } peers { permanent_uuid: "50f6ea8e718846c89f10834eacae7d3b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41793 } }
I20260806 13:09:00.578166 1296 ts_tablet_manager.cc:1428] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:09:00.578187 1060 heartbeater.cc:503] Master 127.13.224.126:37985 was elected leader, sending a full tablet report...
W20260806 13:09:00.592033 1061 tablet.cc:2369] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 13:09:00.669423 881 tablet.cc:2369] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 13:09:00 dist-test-slave-q9h9 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021740, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for HTTP/127.13.224.84@KRBTEST.COM
I20260806 13:09:00.752779 1307 raft_consensus.cc:482] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:09:00.752941 1307 raft_consensus.cc:504] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abf6d9751f844097a31c81cb12f7a8eb" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37451 } } peers { permanent_uuid: "3c6061244e894e78af9d55264728a186" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35365 } } peers { permanent_uuid: "50f6ea8e718846c89f10834eacae7d3b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41793 } }
I20260806 13:09:00.753443 1307 leader_election.cc:288] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abf6d9751f844097a31c81cb12f7a8eb (127.13.224.66:37451), 50f6ea8e718846c89f10834eacae7d3b (127.13.224.67:41793)
I20260806 13:09:00.763167 1295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.754346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:57087 (local address 127.13.224.66:37451)
0806 13:09:00.754402 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:00.754404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:00.758713 (+ 4309us) server_negotiation.cc:407] Connection header received
0806 13:09:00.758837 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:00.758839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:00.758850 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:00.758880 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:00.759311 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.760067 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.761899 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.762549 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.762600 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:00.762609 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:00.763044 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 13:09:00.763092 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 13:09:00.765601 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.754005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:37451 (local address 127.13.224.65:57087)
0806 13:09:00.758636 (+ 4631us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:00.758649 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:00.758757 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:00.758955 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:00.758956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:00.758958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:00.759092 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:00.759095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.760172 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.760173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:00.761793 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:00.761797 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.762718 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.762719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:00.762947 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:00.762952 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:09:00.765367 (+ 2415us) client_negotiation.cc:240] Negotiation successful
0806 13:09:00.765419 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4608,"thread_start_us":1220,"threads_started":1}
W20260806 13:09:00.766230 1245 tablet.cc:2369] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:09:00.769955 1292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.754904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:47393 (local address 127.13.224.67:41793)
0806 13:09:00.754974 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:00.754976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:00.766592 (+ 11616us) server_negotiation.cc:407] Connection header received
0806 13:09:00.766699 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:00.766700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:00.766711 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:00.766740 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:00.766925 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.767635 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.768953 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.769581 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.769630 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:00.769639 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:00.769846 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 13:09:00.769887 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 13:09:00.770469 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.754536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:41793 (local address 127.13.224.65:47393)
0806 13:09:00.766531 (+ 11995us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:00.766543 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:00.766622 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:00.766817 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:00.766817 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:00.766819 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:00.766850 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:00.766851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.767733 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.767735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:00.768848 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:00.768850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.769696 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.769697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:00.769769 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:00.769772 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:09:00.770358 (+ 586us) client_negotiation.cc:240] Negotiation successful
0806 13:09:00.770400 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11984,"thread_start_us":161,"threads_started":1}
I20260806 13:09:00.770923 998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99c8ecfae533435e9f6f76e883414102" candidate_uuid: "3c6061244e894e78af9d55264728a186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abf6d9751f844097a31c81cb12f7a8eb" is_pre_election: true
I20260806 13:09:00.771080 1181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99c8ecfae533435e9f6f76e883414102" candidate_uuid: "3c6061244e894e78af9d55264728a186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50f6ea8e718846c89f10834eacae7d3b" is_pre_election: true
I20260806 13:09:00.771168 998 raft_consensus.cc:2454] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c6061244e894e78af9d55264728a186 in term 0.
I20260806 13:09:00.771236 1181 raft_consensus.cc:2454] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c6061244e894e78af9d55264728a186 in term 0.
I20260806 13:09:00.771447 729 leader_election.cc:302] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c6061244e894e78af9d55264728a186, abf6d9751f844097a31c81cb12f7a8eb; no voters:
I20260806 13:09:00.771577 1307 raft_consensus.cc:2781] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:09:00.771617 1307 raft_consensus.cc:482] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:09:00.771637 1307 raft_consensus.cc:3037] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:00.772951 1307 raft_consensus.cc:504] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abf6d9751f844097a31c81cb12f7a8eb" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37451 } } peers { permanent_uuid: "3c6061244e894e78af9d55264728a186" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35365 } } peers { permanent_uuid: "50f6ea8e718846c89f10834eacae7d3b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41793 } }
I20260806 13:09:00.773121 1307 leader_election.cc:288] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [CANDIDATE]: Term 1 election: Requested vote from peers abf6d9751f844097a31c81cb12f7a8eb (127.13.224.66:37451), 50f6ea8e718846c89f10834eacae7d3b (127.13.224.67:41793)
I20260806 13:09:00.773847 1181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99c8ecfae533435e9f6f76e883414102" candidate_uuid: "3c6061244e894e78af9d55264728a186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50f6ea8e718846c89f10834eacae7d3b"
I20260806 13:09:00.773926 1181 raft_consensus.cc:3037] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:00.774143 998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99c8ecfae533435e9f6f76e883414102" candidate_uuid: "3c6061244e894e78af9d55264728a186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abf6d9751f844097a31c81cb12f7a8eb"
I20260806 13:09:00.774219 998 raft_consensus.cc:3037] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:00.775417 998 raft_consensus.cc:2454] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c6061244e894e78af9d55264728a186 in term 1.
I20260806 13:09:00.775691 729 leader_election.cc:302] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c6061244e894e78af9d55264728a186, abf6d9751f844097a31c81cb12f7a8eb; no voters:
I20260806 13:09:00.777271 1181 raft_consensus.cc:2454] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c6061244e894e78af9d55264728a186 in term 1.
I20260806 13:09:00.777527 1307 raft_consensus.cc:2781] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:09:00.777720 1307 raft_consensus.cc:686] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [term 1 LEADER]: Becoming Leader. State: Replica: 3c6061244e894e78af9d55264728a186, State: Running, Role: LEADER
I20260806 13:09:00.777818 1307 consensus_queue.cc:237] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abf6d9751f844097a31c81cb12f7a8eb" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37451 } } peers { permanent_uuid: "3c6061244e894e78af9d55264728a186" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35365 } } peers { permanent_uuid: "50f6ea8e718846c89f10834eacae7d3b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41793 } }
I20260806 13:09:00.778684 608 catalog_manager.cc:5495] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c6061244e894e78af9d55264728a186 (127.13.224.65). New cstate: current_term: 1 leader_uuid: "3c6061244e894e78af9d55264728a186" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abf6d9751f844097a31c81cb12f7a8eb" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c6061244e894e78af9d55264728a186" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50f6ea8e718846c89f10834eacae7d3b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41793 } health_report { overall_health: UNKNOWN } } }
I20260806 13:09:00.836757 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.833986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40606 (local address 127.13.224.65:35365)
0806 13:09:00.834047 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:00.834049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:00.834077 (+ 28us) server_negotiation.cc:407] Connection header received
0806 13:09:00.834174 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:00.834176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:00.834201 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:00.834278 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:09:00.834408 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.834959 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.835779 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:00.836111 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:00.836174 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:00.836371 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:09:00.836523 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:09:00.836636 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:00.836652 (+ 16us) server_negotiation.cc:299] Negotiation successful
0806 13:09:00.836677 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:09:00.839349 998 raft_consensus.cc:1261] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb [term 1 FOLLOWER]: Refusing update from remote peer 3c6061244e894e78af9d55264728a186: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:09:00.839349 1181 raft_consensus.cc:1261] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b [term 1 FOLLOWER]: Refusing update from remote peer 3c6061244e894e78af9d55264728a186: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:09:00.839880 1307 consensus_queue.cc:1035] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abf6d9751f844097a31c81cb12f7a8eb" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 37451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:09:00.840376 1307 consensus_queue.cc:1035] T 99c8ecfae533435e9f6f76e883414102 P 3c6061244e894e78af9d55264728a186 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50f6ea8e718846c89f10834eacae7d3b" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 41793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:09:00.847019 1320 mvcc.cc:205] Tried to move back new op lower bound from 7315545050473295872 to 7315545050226565120. Current Snapshot: MvccSnapshot[applied={T|T < 7315545050473295872}]
I20260806 13:09:00.847189 1324 mvcc.cc:205] Tried to move back new op lower bound from 7315545050473295872 to 7315545050226565120. Current Snapshot: MvccSnapshot[applied={T|T < 7315545050473295872}]
I20260806 13:09:00.849112 1321 mvcc.cc:205] Tried to move back new op lower bound from 7315545050473295872 to 7315545050226565120. Current Snapshot: MvccSnapshot[applied={T|T < 7315545050473295872}]
I20260806 13:09:00.852036 608 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:09:00.852159 608 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:09:00.852973 608 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5892fd46bbdb4a40bf3b2af26dd5433a: Sending DeleteTablet for 3 replicas of tablet 99c8ecfae533435e9f6f76e883414102
I20260806 13:09:00.853451 776 tablet_service.cc:1514] Processing DeleteTablet for tablet 99c8ecfae533435e9f6f76e883414102 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:00 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:40598
I20260806 13:09:00.853883 947 tablet_service.cc:1514] Processing DeleteTablet for tablet 99c8ecfae533435e9f6f76e883414102 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:00 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:59296
I20260806 13:09:00.854462 1133 tablet_service.cc:1514] Processing DeleteTablet for tablet 99c8ecfae533435e9f6f76e883414102 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:00 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:37870
I20260806 13:09:00.855043 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 664
I20260806 13:09:00.862108 1334 tablet_replica.cc:330] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb: stopping tablet replica
I20260806 13:09:00.862286 1335 tablet_replica.cc:330] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b: stopping tablet replica
I20260806 13:09:00.862473 1335 raft_consensus.cc:2229] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:09:00.862651 1334 raft_consensus.cc:2229] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:09:00.862839 1335 raft_consensus.cc:2258] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:09:00.862982 1334 raft_consensus.cc:2258] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:09:00.863685 1334 ts_tablet_manager.cc:1905] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:09:00.864091 1335 ts_tablet_manager.cc:1905] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:09:00.865821 1335 ts_tablet_manager.cc:1918] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:09:00.867625 1334 ts_tablet_manager.cc:1918] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:09:00.867755 1335 log.cc:1196] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/99c8ecfae533435e9f6f76e883414102
I20260806 13:09:00.868098 1335 ts_tablet_manager.cc:1939] T 99c8ecfae533435e9f6f76e883414102 P 50f6ea8e718846c89f10834eacae7d3b: Deleting consensus metadata
W20260806 13:09:00.868149 590 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:35365 (error 108)
I20260806 13:09:00.868265 1334 log.cc:1196] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/99c8ecfae533435e9f6f76e883414102
I20260806 13:09:00.868595 1334 ts_tablet_manager.cc:1939] T 99c8ecfae533435e9f6f76e883414102 P abf6d9751f844097a31c81cb12f7a8eb: Deleting consensus metadata
I20260806 13:09:00.868806 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 886
I20260806 13:09:00.871268 591 catalog_manager.cc:4841] TS 50f6ea8e718846c89f10834eacae7d3b (127.13.224.67:41793): tablet 99c8ecfae533435e9f6f76e883414102 (table test-table [id=65d666a4e11b4a7ea2b278f6ccc62351]) successfully deleted
I20260806 13:09:00.871393 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.871279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:35365 (local address 127.0.0.1:40616)
0806 13:09:00.871327 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:00.871364 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:35365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 13:09:00.873070 590 catalog_manager.cc:4584] TS 3c6061244e894e78af9d55264728a186 (127.13.224.65:35365): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 99c8ecfae533435e9f6f76e883414102: Network error: Client connection negotiation failed: client connection to 127.13.224.65:35365: connect: Connection refused (error 111)
I20260806 13:09:00.877308 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.877218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:37451 (local address 127.0.0.1:59312)
0806 13:09:00.877258 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:00.877278 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.66:37451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 13:09:00.877363 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 1063
I20260806 13:09:00.920958 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.920795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:37451 (local address 127.0.0.1:59326)
0806 13:09:00.920862 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:00.920905 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.66:37451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 13:09:00.921357 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.921261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:37451 (local address 127.0.0.1:59340)
0806 13:09:00.921304 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:00.921331 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.66:37451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 13:09:00.925545 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.925451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:35365 (local address 127.0.0.1:40628)
0806 13:09:00.925491 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:00.925515 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:35365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 13:09:00.925880 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:00.925807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:35365 (local address 127.0.0.1:40634)
0806 13:09:00.925839 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:00.925858 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:35365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 13:09:00.928979 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 565
I20260806 13:09:00.937270 14209 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 13:09:01.283202 14209 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 13:09:01.283298 14209 mini_ranger.cc:66] Stopping Ranger...
I20260806 13:09:01.626655 14209 mini_ranger.cc:68] Stopped Ranger
2026-08-06 13:09:01.626 UTC [31662] LOG: received smart shutdown request
2026-08-06 13:09:01.632 UTC [31662] LOG: background worker "logical replication launcher" (PID 31675) exited with exit code 1
2026-08-06 13:09:01.633 UTC [31667] LOG: shutting down
2026-08-06 13:09:01.753 UTC [31662] LOG: database system is shut down
2026-08-06T13:09:01Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (45800 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:09:01 dist-test-slave-q9h9 krb5kdc[1349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:09:01 dist-test-slave-q9h9 krb5kdc[1349](info): set up 2 sockets
Aug 06 13:09:01 dist-test-slave-q9h9 krb5kdc[1349](info): commencing operation
krb5kdc: starting...
W20260806 13:09:02.949615 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:09:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:09:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:09:03.060480 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:34007
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:33849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:34007,127.13.224.125:35683,127.13.224.124:42493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:03.138962 1368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:03.139145 1368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:03.139165 1368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:03.140200 1368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:09:03.140232 1368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:03.140244 1368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:03.140255 1368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:09:03.140272 1368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:09:03.141654 1368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:33849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:34007,127.13.224.125:35683,127.13.224.124:42493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:34007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:03.141830 1368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:03.141988 1368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:03.146380 1375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:03.146441 1368 server_base.cc:1034] running on GCE node
W20260806 13:09:03.146389 1377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:03.146379 1374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:03.146867 1368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:03.147109 1368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:03.148243 1368 hybrid_clock.cc:648] HybridClock initialized: now 1786021743148229 us; error 32 us; skew 500 ppm
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:03.150496 1368 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:09:03.150880 1368 webserver.cc:460] Webserver started at http://127.13.224.126:33053/ using document root <none> and password file <none>
I20260806 13:09:03.151081 1368 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:03.151129 1368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:03.151249 1368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:09:03.152225 1368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "1d95984243274e56a0de7ec96212be13"
format_stamp: "Formatted at 2026-08-06 13:09:03 on dist-test-slave-q9h9"
server_key: "66418b7fb3c2f7efa8eb377e79f5c60b"
server_key_iv: "ee302f6603faf863d8a81332e1e433c8"
server_key_version: "encryptionkey@0"
I20260806 13:09:03.152539 1368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "1d95984243274e56a0de7ec96212be13"
format_stamp: "Formatted at 2026-08-06 13:09:03 on dist-test-slave-q9h9"
server_key: "66418b7fb3c2f7efa8eb377e79f5c60b"
server_key_iv: "ee302f6603faf863d8a81332e1e433c8"
server_key_version: "encryptionkey@0"
I20260806 13:09:03.153683 1368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 13:09:03.154349 1384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:03.154488 1368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:03.154560 1368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "1d95984243274e56a0de7ec96212be13"
format_stamp: "Formatted at 2026-08-06 13:09:03 on dist-test-slave-q9h9"
server_key: "66418b7fb3c2f7efa8eb377e79f5c60b"
server_key_iv: "ee302f6603faf863d8a81332e1e433c8"
server_key_version: "encryptionkey@0"
I20260806 13:09:03.154626 1368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:03.162686 1368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:03.164170 1368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:03.164291 1368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:03.167889 1368 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:34007
I20260806 13:09:03.167917 1453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:34007 every 8 connection(s)
I20260806 13:09:03.168313 1368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:09:03.169087 1455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:03.170457 1455 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:09:03.174384 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 1368
I20260806 13:09:03.174487 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:09:03.174669 14209 external_mini_cluster.cc:1428] Setting key 4c6ba15599e8ddc582c11d5453dfec21
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:03.179054 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.171007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:34007 (local address 127.0.0.1:41754)
0806 13:09:03.171154 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.171185 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:03.171580 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:03.171789 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:03.171791 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:03.172000 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:03.172147 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.172151 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.173052 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.173053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.173704 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.173706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.174159 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.174160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.174217 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.174659 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:03.174666 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:03.176146 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:03.177788 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.177795 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.177796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.177972 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.178223 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.178223 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.178223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.178306 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.178658 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:03.178661 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:03.178706 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 13:09:03.178769 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 13:09:03.178880 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260806 13:09:03.179060 1461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.171041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41754 (local address 127.13.224.126:34007)
0806 13:09:03.171187 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.171193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.171261 (+ 68us) server_negotiation.cc:407] Connection header received
0806 13:09:03.171683 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.171685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.171701 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.171781 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.172208 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.172979 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.173768 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.174093 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.174162 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.176222 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.176228 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.176229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.176234 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.177718 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.178056 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.178059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.178060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.178165 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.178358 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.178358 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.178359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.178596 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.178681 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.178809 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.178958 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260806 13:09:03.179706 1455 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:09:03.179980 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.179885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:35683 (local address 127.0.0.1:56544)
0806 13:09:03.179924 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.179962 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:35683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 13:09:03.180099 1388 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.224.125:35683: connect: Connection refused (error 111)
I20260806 13:09:03.180439 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.180366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:35683 (local address 127.0.0.1:56558)
0806 13:09:03.180399 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.180422 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:35683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 13:09:03.180527 1455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.125:35683: Network error: Client connection negotiation failed: client connection to 127.13.224.125:35683: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.13.224.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.125.keytab.
Entry for principal kudu/127.13.224.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.125.keytab.
I20260806 13:09:03.196151 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.125:35683
--webserver_interface=127.13.224.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.125.keytab
--principal=kudu/127.13.224.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:33849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:34007,127.13.224.125:35683,127.13.224.124:42493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:09:03.229717 1455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } attempt: 1
I20260806 13:09:03.230247 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.230064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:35683 (local address 127.0.0.1:56574)
0806 13:09:03.230145 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.230199 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:35683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 13:09:03.230585 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.230496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:35683 (local address 127.0.0.1:56582)
0806 13:09:03.230536 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.230564 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:35683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 13:09:03.230684 1455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.125:35683: Network error: Client connection negotiation failed: client connection to 127.13.224.125:35683: connect: Connection refused (error 111)
W20260806 13:09:03.274729 1466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:03.274916 1466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:03.274936 1466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:03.276013 1466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:09:03.276051 1466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:03.276064 1466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:03.276075 1466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:09:03.276086 1466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:09:03.277486 1466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:33849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:34007,127.13.224.125:35683,127.13.224.124:42493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.125.keytab
--principal=kudu/127.13.224.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.125:35683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.224.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:03.277701 1466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:03.277889 1466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:03.282301 1477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:03.282310 1479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:03.282392 1466 server_base.cc:1034] running on GCE node
W20260806 13:09:03.282310 1476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:03.282739 1466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:03.282987 1466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:03.284122 1466 hybrid_clock.cc:648] HybridClock initialized: now 1786021743284105 us; error 33 us; skew 500 ppm
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:03.286525 1466 init.cc:377] Logged in from keytab as kudu/127.13.224.125@KRBTEST.COM (short username kudu)
I20260806 13:09:03.286928 1466 webserver.cc:460] Webserver started at http://127.13.224.125:33825/ using document root <none> and password file <none>
I20260806 13:09:03.287137 1466 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:03.287197 1466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:03.287315 1466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:09:03.288234 1466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/data/instance:
uuid: "99ffe5e3e0cd40c2901707f56ccdabfb"
format_stamp: "Formatted at 2026-08-06 13:09:03 on dist-test-slave-q9h9"
server_key: "29812918bb65181562f5b05f072cc015"
server_key_iv: "de935649a5ee77a5ba2a1e1ac1d72cb6"
server_key_version: "encryptionkey@0"
I20260806 13:09:03.288558 1466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/wal/instance:
uuid: "99ffe5e3e0cd40c2901707f56ccdabfb"
format_stamp: "Formatted at 2026-08-06 13:09:03 on dist-test-slave-q9h9"
server_key: "29812918bb65181562f5b05f072cc015"
server_key_iv: "de935649a5ee77a5ba2a1e1ac1d72cb6"
server_key_version: "encryptionkey@0"
I20260806 13:09:03.289743 1466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 13:09:03.290443 1488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:03.290575 1466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:03.290656 1466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/wal
uuid: "99ffe5e3e0cd40c2901707f56ccdabfb"
format_stamp: "Formatted at 2026-08-06 13:09:03 on dist-test-slave-q9h9"
server_key: "29812918bb65181562f5b05f072cc015"
server_key_iv: "de935649a5ee77a5ba2a1e1ac1d72cb6"
server_key_version: "encryptionkey@0"
I20260806 13:09:03.290721 1466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:03.298818 1455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } attempt: 2
I20260806 13:09:03.299173 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.299065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:35683 (local address 127.0.0.1:56588)
0806 13:09:03.299114 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.299147 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:35683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 13:09:03.299455 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.299379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:35683 (local address 127.0.0.1:56600)
0806 13:09:03.299414 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.299437 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:35683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 13:09:03.299532 1455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.125:35683: Network error: Client connection negotiation failed: client connection to 127.13.224.125:35683: connect: Connection refused (error 111)
I20260806 13:09:03.304158 1466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:03.305644 1466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:03.305774 1466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:03.309370 1466 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.125:35683
I20260806 13:09:03.309399 1556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.125:35683 every 8 connection(s)
I20260806 13:09:03.309690 1466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/data/info.pb
I20260806 13:09:03.310220 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 1466
I20260806 13:09:03.310341 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-1/wal/instance
I20260806 13:09:03.310514 14209 external_mini_cluster.cc:1428] Setting key 03ab0332914f323f48df9a752d06ea3f
I20260806 13:09:03.310616 1558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:03.312455 1558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.125@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.224.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.124@KRBTEST.COM" created.
I20260806 13:09:03.320899 1461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.313071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.13.224.126:34007)
0806 13:09:03.313126 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.313128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.313280 (+ 152us) server_negotiation.cc:407] Connection header received
0806 13:09:03.313690 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.313691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.313706 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.313773 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.314370 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.315114 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.316033 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.316348 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.316417 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.318574 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.318576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.318577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.318585 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.319783 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.320118 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.320119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.320119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.320176 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.320390 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.320391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.320391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.320484 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.320526 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.320746 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.320809 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 13:09:03.321069 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.313027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:34007 (local address 127.0.0.1:41766)
0806 13:09:03.313166 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.313205 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:03.313604 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:03.313827 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:03.313830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:03.314094 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:03.314280 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.314284 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.315205 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.315206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.315953 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.315955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.316426 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.316427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.316519 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.316996 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:03.317003 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:03.318480 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:03.319860 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.319867 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.319868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.320066 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.320221 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.320221 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.320221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.320315 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.320531 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:03.320536 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:03.320647 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 13:09:03.320728 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 13:09:03.320876 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
I20260806 13:09:03.321688 1558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.125@KRBTEST.COM for kudu/127.13.224.125@KRBTEST.COM
I20260806 13:09:03.328702 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.321919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:35683 (local address 127.0.0.1:56612)
0806 13:09:03.321962 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.321971 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:03.322000 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:03.322239 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:03.322240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:03.322373 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:03.322428 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.322430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.323575 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.323576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.324040 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.324041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.324662 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.324663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.324712 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.325042 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:03.325047 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:03.326329 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:03.327812 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.327814 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.327814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.328002 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.328169 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.328170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.328170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.328228 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.328451 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:03.328452 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:03.328496 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 13:09:03.328565 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 13:09:03.328643 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 13:09:03.328814 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.321971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56612 (local address 127.13.224.125:35683)
0806 13:09:03.322111 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.322116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.322133 (+ 17us) server_negotiation.cc:407] Connection header received
0806 13:09:03.322154 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.322156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.322166 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.322223 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.322496 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.323499 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.324112 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.324585 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.324654 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.326404 (+ 1750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.326411 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.326412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.326417 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.327731 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.328073 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.328075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.328075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.328120 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.328286 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.328286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.328287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.328395 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.328448 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.328590 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.328704 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":114,"thread_start_us":91,"threads_started":1}
I20260806 13:09:03.329319 1558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:09:03.329608 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.329488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42493 (local address 127.0.0.1:39902)
0806 13:09:03.329549 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.329591 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 13:09:03.329730 1495 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
I20260806 13:09:03.330304 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.330216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42493 (local address 127.0.0.1:39916)
0806 13:09:03.330263 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.330288 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 13:09:03.330400 1558 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:42493: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.124.keytab.
Entry for principal kudu/127.13.224.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.124.keytab.
I20260806 13:09:03.333504 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.124:42493
--webserver_interface=127.13.224.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.124.keytab
--principal=kudu/127.13.224.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:33849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:34007,127.13.224.125:35683,127.13.224.124:42493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:09:03.379617 1558 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } attempt: 1
I20260806 13:09:03.380087 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.379929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42493 (local address 127.0.0.1:39926)
0806 13:09:03.379995 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.380046 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 13:09:03.380443 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.380352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42493 (local address 127.0.0.1:39930)
0806 13:09:03.380395 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.380423 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 13:09:03.380535 1558 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:42493: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
I20260806 13:09:03.390666 1455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } attempt: 3
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for kudu/127.13.224.125@KRBTEST.COM
I20260806 13:09:03.397209 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.390974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:35683 (local address 127.0.0.1:56626)
0806 13:09:03.391026 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.391036 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:03.391081 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:03.391211 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:03.391211 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:03.391363 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:03.391432 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.391434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.392057 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.392059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.392569 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.392571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.392956 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.392957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.393025 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.393424 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:03.393429 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:03.394651 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:03.396305 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.396306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.396307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.396463 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.396649 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.396649 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.396649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.396698 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.396953 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:03.396954 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:03.397005 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 13:09:03.397058 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 13:09:03.397134 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 13:09:03.397269 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.391043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56626 (local address 127.13.224.125:35683)
0806 13:09:03.391101 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.391103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.391113 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:09:03.391139 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.391140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.391153 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.391198 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.391503 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.391987 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.392637 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.392883 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.392937 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.394731 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.394733 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.394733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.394748 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.396214 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.396524 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.396525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.396526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.396593 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.396752 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.396752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.396752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.396890 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.396949 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.397094 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.397177 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 13:09:03.397609 1455 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:09:03.397954 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.397862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42493 (local address 127.0.0.1:39946)
0806 13:09:03.397906 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.397934 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 13:09:03.398275 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.398197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42493 (local address 127.0.0.1:39954)
0806 13:09:03.398232 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.398255 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 13:09:03.398347 1455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:42493: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
W20260806 13:09:03.414325 1567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:03.414505 1567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:03.414535 1567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:03.415668 1567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:09:03.415707 1567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:03.415727 1567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:03.415743 1567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:09:03.415759 1567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:09:03.417218 1567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:33849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:34007,127.13.224.125:35683,127.13.224.124:42493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.124.keytab
--principal=kudu/127.13.224.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.124:42493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.224.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:03.417464 1567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:03.417662 1567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:03.422173 1573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:03.422164 1574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:03.422166 1577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:03.422240 1567 server_base.cc:1034] running on GCE node
I20260806 13:09:03.422569 1567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:03.422855 1567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:03.423995 1567 hybrid_clock.cc:648] HybridClock initialized: now 1786021743423982 us; error 23 us; skew 500 ppm
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:03.426247 1567 init.cc:377] Logged in from keytab as kudu/127.13.224.124@KRBTEST.COM (short username kudu)
I20260806 13:09:03.426631 1567 webserver.cc:460] Webserver started at http://127.13.224.124:33871/ using document root <none> and password file <none>
I20260806 13:09:03.426854 1567 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:03.426903 1567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:03.427022 1567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:09:03.427976 1567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/data/instance:
uuid: "1d515feedce3435ca1f02e19917f478c"
format_stamp: "Formatted at 2026-08-06 13:09:03 on dist-test-slave-q9h9"
server_key: "863abb034271c12035cc730a98c46615"
server_key_iv: "defe221fa6d4765e6dba49e60ddcf8a3"
server_key_version: "encryptionkey@0"
I20260806 13:09:03.428300 1567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/wal/instance:
uuid: "1d515feedce3435ca1f02e19917f478c"
format_stamp: "Formatted at 2026-08-06 13:09:03 on dist-test-slave-q9h9"
server_key: "863abb034271c12035cc730a98c46615"
server_key_iv: "defe221fa6d4765e6dba49e60ddcf8a3"
server_key_version: "encryptionkey@0"
I20260806 13:09:03.429478 1567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:09:03.429473 1455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } attempt: 1
I20260806 13:09:03.429797 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.429693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42493 (local address 127.0.0.1:39962)
0806 13:09:03.429742 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.429771 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 13:09:03.430150 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.430054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42493 (local address 127.0.0.1:39976)
0806 13:09:03.430090 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.430116 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 13:09:03.430212 1586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260806 13:09:03.430256 1455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:42493: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
I20260806 13:09:03.430384 1567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:09:03.430464 1567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/wal
uuid: "1d515feedce3435ca1f02e19917f478c"
format_stamp: "Formatted at 2026-08-06 13:09:03 on dist-test-slave-q9h9"
server_key: "863abb034271c12035cc730a98c46615"
server_key_iv: "defe221fa6d4765e6dba49e60ddcf8a3"
server_key_version: "encryptionkey@0"
I20260806 13:09:03.430516 1567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:03.448675 1558 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } attempt: 2
I20260806 13:09:03.449149 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.448996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42493 (local address 127.0.0.1:39986)
0806 13:09:03.449059 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.449108 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 13:09:03.449476 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.449394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42493 (local address 127.0.0.1:39998)
0806 13:09:03.449434 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.449459 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 13:09:03.449560 1558 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:42493: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42493: connect: Connection refused (error 111)
I20260806 13:09:03.455641 1567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:03.457429 1567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:03.457561 1567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:03.461383 1567 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.124:42493
I20260806 13:09:03.461419 1657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.124:42493 every 8 connection(s)
I20260806 13:09:03.461742 1567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/data/info.pb
I20260806 13:09:03.462621 1661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:03.464149 1661 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:09:03.467909 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 1567
I20260806 13:09:03.468006 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786021591455662-14209-0/minicluster-data/master-2/wal/instance
I20260806 13:09:03.468191 14209 external_mini_cluster.cc:1428] Setting key ac109129685beb0a1fe65920b2ee4c3f
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.124@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:03.472730 1461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.464830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41770 (local address 127.13.224.126:34007)
0806 13:09:03.464893 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.464895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.465042 (+ 147us) server_negotiation.cc:407] Connection header received
0806 13:09:03.465508 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.465510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.465524 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.465571 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.466216 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.466686 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.467626 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.467869 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.467937 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.470213 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.470215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.470215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.470223 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.471435 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.471857 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.471858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.471859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.471925 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.472144 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.472145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.472145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.472279 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.472355 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.472561 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.472642 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 13:09:03.472786 1663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.464758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:34007 (local address 127.0.0.1:41770)
0806 13:09:03.464935 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.464971 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:03.465411 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:03.465650 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:03.465653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:03.465904 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:03.466134 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.466139 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.466769 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.466770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.467545 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.467547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.467969 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.467970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.468117 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.468709 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:03.468719 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:03.470129 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:03.471511 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.471518 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.471520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.471780 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.471987 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.471988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.471988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.472087 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.472341 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:03.472344 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:03.472448 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 13:09:03.472523 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 13:09:03.472632 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260806 13:09:03.473294 1661 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.124@KRBTEST.COM
I20260806 13:09:03.476114 1666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.468529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40002 (local address 127.13.224.124:42493)
0806 13:09:03.468676 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.468680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.468830 (+ 150us) server_negotiation.cc:407] Connection header received
0806 13:09:03.468858 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.468861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.468883 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.468948 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.469593 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.470354 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.471066 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.471362 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.471419 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.473800 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.473807 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.473808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.473815 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.475075 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.475383 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.475386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.475386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.475434 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.475597 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.475598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.475599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.475722 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.475868 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.475936 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.476020 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.124@KRBTEST.COM for kudu/127.13.224.125@KRBTEST.COM
I20260806 13:09:03.479421 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.473571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56628 (local address 127.13.224.125:35683)
0806 13:09:03.473638 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.473640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.473647 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:09:03.473670 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.473670 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.473679 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.473718 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.473969 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.474592 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.475208 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.475413 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.475466 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.477058 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.477059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.477059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.477066 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.478300 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.478576 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.478577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.478578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.478629 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.478798 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.478799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.478800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.478915 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.478980 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.479304 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.479362 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 13:09:03.479439 1663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.473499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:35683 (local address 127.0.0.1:56628)
0806 13:09:03.473538 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.473547 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:03.473580 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:03.473736 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:03.473737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:03.473862 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:03.473904 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.473905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.474656 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.474657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.475137 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.475138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.475470 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.475471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.475535 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.475843 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:03.475847 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:03.476986 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:03.478369 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.478370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.478371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.478518 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.478680 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.478680 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.478681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.478734 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.479153 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:03.479155 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:03.479197 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 13:09:03.479270 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 13:09:03.479351 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 13:09:03.479856 1661 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:03.482970 1461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.476831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41780 (local address 127.13.224.126:34007)
0806 13:09:03.476864 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.476866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.476873 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:09:03.476909 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.476910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.476918 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.476968 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.477378 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.477803 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.478467 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.478640 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.478690 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.480806 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.480807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.480807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.480814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.481997 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.482306 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.482307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.482307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.482359 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.482546 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.482547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.482547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.482664 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.482791 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.482861 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.482915 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.124@KRBTEST.COM for kudu/127.13.224.124@KRBTEST.COM
I20260806 13:09:03.485934 1663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.480514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42493 (local address 127.0.0.1:40016)
0806 13:09:03.480591 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.480599 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:03.480636 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:03.480747 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:03.480748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:03.480845 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:03.480870 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.480871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.481444 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.481445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.481881 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.481882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.482246 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.482247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.482290 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.482595 (+ 305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:03.482599 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:03.483706 (+ 1107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:03.485037 (+ 1331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.485038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.485038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.485234 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.485403 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.485403 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.485403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.485454 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.485657 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:03.485658 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:03.485700 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 13:09:03.485786 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 13:09:03.485877 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 13:09:03.486050 1666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.480556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.13.224.124:42493)
0806 13:09:03.480590 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.480592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.480629 (+ 37us) server_negotiation.cc:407] Connection header received
0806 13:09:03.480670 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.480671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.480680 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.480738 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.480932 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.481380 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.481945 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.482190 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.482249 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.483780 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.483781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.483781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.483788 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.484966 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.485297 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.485298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.485298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.485352 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.485498 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.485499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.485499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.485605 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.485661 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.485805 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.485938 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":17}
I20260806 13:09:03.487221 1661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c: Bootstrap starting.
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.125@KRBTEST.COM
I20260806 13:09:03.488097 1661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:03.488408 1661 log.cc:824] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:03.489212 1661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c: No bootstrap required, opened a new log
I20260806 13:09:03.490245 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.483647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56644 (local address 127.13.224.125:35683)
0806 13:09:03.483677 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.483678 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.483686 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:09:03.483746 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.483747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.483755 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.483800 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.484196 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.484613 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.485296 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.485582 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.485639 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.487911 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.487913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.487913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.487919 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.489263 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.489618 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.489619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.489619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.489670 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.489843 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.489844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.489844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.489928 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.490057 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.490127 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.490185 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 13:09:03.491075 1661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } }
I20260806 13:09:03.491202 1661 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:03.491235 1661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d515feedce3435ca1f02e19917f478c, State: Initialized, Role: FOLLOWER
I20260806 13:09:03.491346 1661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } }
I20260806 13:09:03.491585 1669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } } }
I20260806 13:09:03.491636 1661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:09:03.491686 1669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:09:03.491899 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 13:09:03.493464 1686 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:09:03.493571 1686 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:09:03.493731 1567 master_runner.cc:186] Error getting master registration for 127.13.224.126:34007: OK, instance_id { permanent_uuid: "1d95984243274e56a0de7ec96212be13" instance_seqno: 1786021743166902 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 13:09:03.496542 1567 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 13:09:03.500778 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.493412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.13.224.125:35683)
0806 13:09:03.493447 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.493449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.493898 (+ 449us) server_negotiation.cc:407] Connection header received
0806 13:09:03.493951 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.493952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.493961 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.494005 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.494654 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.495354 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.496341 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.496624 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.496660 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.498159 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.498160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.498161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.498167 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.499607 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.500009 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.500010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.500010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.500066 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.500272 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.500273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.500274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.500382 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.500441 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.500626 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.500695 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:09:03.501056 1666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.493736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40020 (local address 127.13.224.124:42493)
0806 13:09:03.493770 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.493772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.494076 (+ 304us) server_negotiation.cc:407] Connection header received
0806 13:09:03.494157 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.494158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.494167 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.494194 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.494774 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.495438 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.496341 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.496625 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.496662 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.498271 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.498272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.498273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.498279 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.499742 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.500127 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.500128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.500129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.500183 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.500387 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.500388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.500388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.500531 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.500613 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.500905 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.500974 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:09:03.504441 1461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.493367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41784 (local address 127.13.224.126:34007)
0806 13:09:03.493446 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.493448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.497519 (+ 4071us) server_negotiation.cc:407] Connection header received
0806 13:09:03.497615 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.497616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.497625 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.497654 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.498236 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.499636 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.500518 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.500785 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.500825 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.502222 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.502223 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.502224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.502230 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.503474 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.503771 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.503772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.503772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.503824 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.503984 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.503985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.503985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.504081 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.504155 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.504289 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.504354 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 13:09:03.505376 1455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } attempt: 2
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for kudu/127.13.224.124@KRBTEST.COM
I20260806 13:09:03.511464 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.505595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42493 (local address 127.0.0.1:40036)
0806 13:09:03.505642 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.505652 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:03.505691 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:03.505898 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:03.505898 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:03.506003 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:03.506059 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.506060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.506861 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.506861 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.507302 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.507303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.507691 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.507691 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.507722 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.508054 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:03.508059 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:03.509169 (+ 1110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:03.510710 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.510711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.510711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.510870 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.511035 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.511035 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.511036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.511077 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.511279 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:03.511279 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:03.511309 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 13:09:03.511351 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 13:09:03.511408 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 13:09:03.511530 1666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.505749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40036 (local address 127.13.224.124:42493)
0806 13:09:03.505797 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.505799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.505807 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:09:03.505825 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.505826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.505834 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.505894 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.506120 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.506797 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.507356 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.507636 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.507696 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.509269 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.509271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.509271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.509277 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.510636 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.510930 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.510931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.510931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.510983 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.511111 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.511112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.511112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.511218 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.511287 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.511384 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.511449 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 13:09:03.512931 1455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13: Bootstrap starting.
I20260806 13:09:03.513648 1455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:03.513947 1455 log.cc:824] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:03.514773 1455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13: No bootstrap required, opened a new log
I20260806 13:09:03.516712 1455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } }
I20260806 13:09:03.516849 1455 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:03.516892 1455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d95984243274e56a0de7ec96212be13, State: Initialized, Role: FOLLOWER
I20260806 13:09:03.517006 1455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } }
I20260806 13:09:03.517242 1698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } } }
I20260806 13:09:03.517304 1455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:09:03.517319 1698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 13:09:03.519243 1711 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:09:03.519344 1711 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:09:03.519973 1368 master_runner.cc:186] Error getting master registration for 127.13.224.125:35683: OK, instance_id { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" instance_seqno: 1786021743308380 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 13:09:03.522145 1368 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 13:09:03.540706 1558 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } attempt: 3
Aug 06 13:09:03 dist-test-slave-q9h9 krb5kdc[1349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021743, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.125@KRBTEST.COM for kudu/127.13.224.124@KRBTEST.COM
I20260806 13:09:03.548866 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.540984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42493 (local address 127.0.0.1:40046)
0806 13:09:03.541034 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:03.541044 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:03.541080 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:03.541236 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:03.541237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:03.541433 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:03.541487 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.541489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.542437 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.542438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.543174 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:03.543176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.543629 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.543631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:03.543696 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.544159 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:03.544165 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:03.545681 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:03.547432 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.547433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.547434 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.547675 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.548216 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:03.548217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:03.548217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:03.548296 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:03.548551 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:03.548552 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:03.548597 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 13:09:03.548663 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 13:09:03.548759 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 13:09:03.548887 1666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:03.541068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40046 (local address 127.13.224.124:42493)
0806 13:09:03.541117 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:03.541120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:03.541128 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:09:03.541150 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:03.541151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:03.541161 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:03.541223 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:03.541560 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.542363 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.543254 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:03.543542 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:03.543606 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:03.545780 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:03.545782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:03.545782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:03.545789 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:03.547350 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.548081 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.548082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.548083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.548146 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:03.548361 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:03.548362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:03.548362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:03.548492 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:03.548556 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:03.548694 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 13:09:03.548785 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 13:09:03.550968 1558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb: Bootstrap starting.
I20260806 13:09:03.551714 1558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:03.552021 1558 log.cc:824] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:03.552889 1558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb: No bootstrap required, opened a new log
I20260806 13:09:03.554817 1558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } }
I20260806 13:09:03.554951 1558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:03.554980 1558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ffe5e3e0cd40c2901707f56ccdabfb, State: Initialized, Role: FOLLOWER
I20260806 13:09:03.555181 1558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } }
I20260806 13:09:03.555473 1714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } } }
I20260806 13:09:03.555523 1558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [sys.catalog]: configured and running, proceeding with master startup.
W20260806 13:09:03.556332 1497 tablet.cc:2369] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 13:09:03.558259 1730 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:09:03.558375 1730 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:09:03.555603 1714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:09:03.560106 1466 master_runner.cc:263] No leader master found from master 99ffe5e3e0cd40c2901707f56ccdabfb
I20260806 13:09:03.560151 1466 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 13:09:03.613551 1714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:09:03.613677 1714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } }
I20260806 13:09:03.613906 1714 leader_election.cc:288] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d95984243274e56a0de7ec96212be13 (127.13.224.126:34007), 1d515feedce3435ca1f02e19917f478c (127.13.224.124:42493)
I20260806 13:09:03.614387 1622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d515feedce3435ca1f02e19917f478c" is_pre_election: true
I20260806 13:09:03.614548 1622 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99ffe5e3e0cd40c2901707f56ccdabfb in term 0.
I20260806 13:09:03.614885 1495 leader_election.cc:302] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d515feedce3435ca1f02e19917f478c, 99ffe5e3e0cd40c2901707f56ccdabfb; no voters:
I20260806 13:09:03.615078 1714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:09:03.615135 1714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:09:03.615156 1714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:03.616145 1714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } }
I20260806 13:09:03.616258 1714 leader_election.cc:288] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [CANDIDATE]: Term 1 election: Requested vote from peers 1d95984243274e56a0de7ec96212be13 (127.13.224.126:34007), 1d515feedce3435ca1f02e19917f478c (127.13.224.124:42493)
I20260806 13:09:03.616662 1417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d95984243274e56a0de7ec96212be13"
I20260806 13:09:03.616801 1417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:03.616662 1418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d95984243274e56a0de7ec96212be13" is_pre_election: true
I20260806 13:09:03.617177 1622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d515feedce3435ca1f02e19917f478c"
I20260806 13:09:03.617251 1622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:03.617709 1417 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99ffe5e3e0cd40c2901707f56ccdabfb in term 1.
I20260806 13:09:03.618374 1489 leader_election.cc:302] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d95984243274e56a0de7ec96212be13, 99ffe5e3e0cd40c2901707f56ccdabfb; no voters:
I20260806 13:09:03.618636 1622 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99ffe5e3e0cd40c2901707f56ccdabfb in term 1.
I20260806 13:09:03.618882 1714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:09:03.619062 1714 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [term 1 LEADER]: Becoming Leader. State: Replica: 99ffe5e3e0cd40c2901707f56ccdabfb, State: Running, Role: LEADER
I20260806 13:09:03.619179 1714 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } }
I20260806 13:09:03.619547 1714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99ffe5e3e0cd40c2901707f56ccdabfb. Latest consensus state: current_term: 1 leader_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } } }
I20260806 13:09:03.619614 1714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:03.620401 1736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:09:03.620667 1736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:09:03.623198 1622 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [term 1 FOLLOWER]: Refusing update from remote peer 99ffe5e3e0cd40c2901707f56ccdabfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:09:03.623529 1714 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:09:03.624861 1669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99ffe5e3e0cd40c2901707f56ccdabfb. Latest consensus state: current_term: 1 leader_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } } }
I20260806 13:09:03.624963 1669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:09:03.629554 1417 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [term 1 FOLLOWER]: Refusing update from remote peer 99ffe5e3e0cd40c2901707f56ccdabfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:09:03.630002 1714 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:09:03.632277 1698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99ffe5e3e0cd40c2901707f56ccdabfb. Latest consensus state: current_term: 1 leader_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } } }
I20260806 13:09:03.632386 1698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:09:03.634505 1733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } } }
I20260806 13:09:03.635023 1698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } } }
I20260806 13:09:03.635102 1698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:09:03.635485 1733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:03.635628 1737 mvcc.cc:205] Tried to move back new op lower bound from 7315545061878222848 to 7315545061864665088. Current Snapshot: MvccSnapshot[applied={T|T < 7315545061878222848}]
I20260806 13:09:03.635977 1741 mvcc.cc:205] Tried to move back new op lower bound from 7315545061878222848 to 7315545061864665088. Current Snapshot: MvccSnapshot[applied={T|T < 7315545061878222848}]
I20260806 13:09:03.637652 1733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } } }
I20260806 13:09:03.637728 1733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:03.638182 1669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95984243274e56a0de7ec96212be13" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34007 } } peers { permanent_uuid: "99ffe5e3e0cd40c2901707f56ccdabfb" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 35683 } } peers { permanent_uuid: "1d515feedce3435ca1f02e19917f478c" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42493 } } }
I20260806 13:09:03.638259 1669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:09:03.638923 1738 mvcc.cc:205] Tried to move back new op lower bound from 7315545061878222848 to 7315545061864665088. Current Snapshot: MvccSnapshot[applied={T|T < 7315545061878222848}]
I20260806 13:09:03.639460 1736 catalog_manager.cc:1347] Generated new cluster ID: 9052be990d4f48ecb726ce5efd3387a2
I20260806 13:09:03.639504 1736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:09:03.663723 1736 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:09:03.664441 1736 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:09:03.668416 1736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99ffe5e3e0cd40c2901707f56ccdabfb: Generated new TSK 0
I20260806 13:09:03.668584 1736 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:09:04.494212 1686 catalog_manager.cc:1259] Loaded cluster ID: 9052be990d4f48ecb726ce5efd3387a2
I20260806 13:09:04.494287 1686 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c: loading cluster ID for follower catalog manager: success
I20260806 13:09:04.495105 1686 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c: acquiring CA information for follower catalog manager: success
I20260806 13:09:04.495422 1686 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1d515feedce3435ca1f02e19917f478c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 13:09:04.519819 1711 catalog_manager.cc:1259] Loaded cluster ID: 9052be990d4f48ecb726ce5efd3387a2
I20260806 13:09:04.519887 1711 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13: loading cluster ID for follower catalog manager: success
I20260806 13:09:04.520712 1711 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13: acquiring CA information for follower catalog manager: success
I20260806 13:09:04.521004 1711 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1d95984243274e56a0de7ec96212be13: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 13:09:04.718317 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 1368
I20260806 13:09:04.730355 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 1466
I20260806 13:09:04.741431 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 1567
2026-08-06T13:09:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2902 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:09:04 dist-test-slave-q9h9 krb5kdc[1759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:09:04 dist-test-slave-q9h9 krb5kdc[1759](info): set up 2 sockets
Aug 06 13:09:04 dist-test-slave-q9h9 krb5kdc[1759](info): commencing operation
krb5kdc: starting...
W20260806 13:09:05.857611 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:09:05 dist-test-slave-q9h9 krb5kdc[1759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:09:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:09:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:09:05.962225 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:44737
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:42837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:44737
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:06.041522 1776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:06.041723 1776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:06.041754 1776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:06.042873 1776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:09:06.042919 1776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:06.042943 1776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:06.042963 1776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:09:06.042981 1776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:09:06.044450 1776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:42837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:44737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:44737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:06.044689 1776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:06.044896 1776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:06.049432 1787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:06.049445 1783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:06.049471 1776 server_base.cc:1034] running on GCE node
W20260806 13:09:06.049444 1785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:06.049822 1776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:06.050129 1776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:06.051272 1776 hybrid_clock.cc:648] HybridClock initialized: now 1786021746051260 us; error 30 us; skew 500 ppm
Aug 06 13:09:06 dist-test-slave-q9h9 krb5kdc[1759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021746, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:06.058041 1776 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:09:06.058458 1776 webserver.cc:460] Webserver started at http://127.13.224.126:36983/ using document root <none> and password file <none>
I20260806 13:09:06.058641 1776 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:06.058688 1776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:06.058813 1776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:09:06.059782 1776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "a7b5905459f142dd8bf6292f5a945e3b"
format_stamp: "Formatted at 2026-08-06 13:09:06 on dist-test-slave-q9h9"
server_key: "008bc9e998e5018ec822a530b5c346f3"
server_key_iv: "b1edeb1aa930869e1352d8bdb2373c05"
server_key_version: "encryptionkey@0"
I20260806 13:09:06.060093 1776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "a7b5905459f142dd8bf6292f5a945e3b"
format_stamp: "Formatted at 2026-08-06 13:09:06 on dist-test-slave-q9h9"
server_key: "008bc9e998e5018ec822a530b5c346f3"
server_key_iv: "b1edeb1aa930869e1352d8bdb2373c05"
server_key_version: "encryptionkey@0"
I20260806 13:09:06.061337 1776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 13:09:06.061942 1796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:06.062093 1776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:09:06.062172 1776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "a7b5905459f142dd8bf6292f5a945e3b"
format_stamp: "Formatted at 2026-08-06 13:09:06 on dist-test-slave-q9h9"
server_key: "008bc9e998e5018ec822a530b5c346f3"
server_key_iv: "b1edeb1aa930869e1352d8bdb2373c05"
server_key_version: "encryptionkey@0"
I20260806 13:09:06.062230 1776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:06.080267 1776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:06.081840 1776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:06.081971 1776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:06.085579 1776 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:44737
I20260806 13:09:06.085628 1859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:44737 every 8 connection(s)
I20260806 13:09:06.085953 1776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:09:06.086827 1860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:06.089080 1860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b: Bootstrap starting.
I20260806 13:09:06.089634 1860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:06.089867 1860 log.cc:824] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:06.090567 1860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b: No bootstrap required, opened a new log
I20260806 13:09:06.092096 1860 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b5905459f142dd8bf6292f5a945e3b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 44737 } }
I20260806 13:09:06.092234 1860 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:06.092264 1860 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7b5905459f142dd8bf6292f5a945e3b, State: Initialized, Role: FOLLOWER
I20260806 13:09:06.092371 1860 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b5905459f142dd8bf6292f5a945e3b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 44737 } }
I20260806 13:09:06.092425 1860 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:09:06.092456 1860 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:09:06.092491 1860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:06.092974 1860 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b5905459f142dd8bf6292f5a945e3b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 44737 } }
I20260806 13:09:06.093070 1860 leader_election.cc:302] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7b5905459f142dd8bf6292f5a945e3b; no voters:
I20260806 13:09:06.093202 1860 leader_election.cc:288] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:09:06.093246 1864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:09:06.093425 1864 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [term 1 LEADER]: Becoming Leader. State: Replica: a7b5905459f142dd8bf6292f5a945e3b, State: Running, Role: LEADER
I20260806 13:09:06.093431 1860 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:09:06.093545 1864 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b5905459f142dd8bf6292f5a945e3b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 44737 } }
I20260806 13:09:06.093924 1866 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7b5905459f142dd8bf6292f5a945e3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b5905459f142dd8bf6292f5a945e3b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 44737 } } }
I20260806 13:09:06.094005 1866 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:06.094141 1867 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7b5905459f142dd8bf6292f5a945e3b. Latest consensus state: current_term: 1 leader_uuid: "a7b5905459f142dd8bf6292f5a945e3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b5905459f142dd8bf6292f5a945e3b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 44737 } } }
I20260806 13:09:06.094230 1867 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:06.094254 1875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:09:06.094767 1875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:09:06.096014 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 1776
I20260806 13:09:06.096122 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:09:06.096318 14209 external_mini_cluster.cc:1428] Setting key 2aa1e3c3b2cf2ba4e2088f1a9fe96cd9
I20260806 13:09:06.099116 1875 catalog_manager.cc:1347] Generated new cluster ID: cd4406f4673b41eab6802931c61858dc
I20260806 13:09:06.099167 1875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:09:06 dist-test-slave-q9h9 krb5kdc[1759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:06.112221 1875 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:09:06.112785 1875 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:09:06.114203 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:06.096760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.13.224.126:44737)
0806 13:09:06.096923 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:06.096932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:06.096961 (+ 29us) server_negotiation.cc:407] Connection header received
0806 13:09:06.097124 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:06.097131 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:06.097173 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:06.097285 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:06.097880 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:06.099003 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:06.106918 (+ 7915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:06.107538 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:06.107664 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:06.110421 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:06.110425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:06.110425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:06.110437 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:06.112471 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:06.112906 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:06.112910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:06.112910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:06.113020 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:06.113239 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:06.113240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:06.113240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:06.113527 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:06.113714 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:06.113783 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 13:09:06.113930 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260806 13:09:06.115150 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:09:06.118988 1875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7b5905459f142dd8bf6292f5a945e3b: Generated new TSK 0
I20260806 13:09:06.119143 1875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:09:06.168841 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 1776
2026-08-06T13:09:06Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1427 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:09:06 dist-test-slave-q9h9 krb5kdc[1897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:09:06 dist-test-slave-q9h9 krb5kdc[1897](info): set up 2 sockets
Aug 06 13:09:06 dist-test-slave-q9h9 krb5kdc[1897](info): commencing operation
krb5kdc: starting...
W20260806 13:09:07.285281 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:09:07 dist-test-slave-q9h9 krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:09:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:09:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:09:07.390929 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:39465
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:39465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:07.470544 1914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:07.470736 1914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:07.470765 1914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:07.471869 1914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:09:07.471908 1914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:07.471927 1914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:07.471943 1914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:09:07.471958 1914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:09:07.473383 1914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:39465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:39465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:07.473616 1914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:07.473814 1914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:07.478258 1919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:07.478431 1914 server_base.cc:1034] running on GCE node
W20260806 13:09:07.478293 1920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:07.478268 1923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:07.478752 1914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:07.479002 1914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:07.480149 1914 hybrid_clock.cc:648] HybridClock initialized: now 1786021747480136 us; error 30 us; skew 500 ppm
Aug 06 13:09:07 dist-test-slave-q9h9 krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021747, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:07.482391 1914 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:09:07.482800 1914 webserver.cc:460] Webserver started at http://127.13.224.126:41351/ using document root <none> and password file <none>
I20260806 13:09:07.482985 1914 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:07.483032 1914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:07.483157 1914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:09:07.484104 1914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "31ba368ccec941f58fd1aa7e6dcb868c"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "5690a8a5ddcc3ce6f1e1717025b4d691"
server_key_iv: "b35f83cc9a7e46d1ce57bf0fd76d4388"
server_key_version: "encryptionkey@0"
I20260806 13:09:07.484428 1914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "31ba368ccec941f58fd1aa7e6dcb868c"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "5690a8a5ddcc3ce6f1e1717025b4d691"
server_key_iv: "b35f83cc9a7e46d1ce57bf0fd76d4388"
server_key_version: "encryptionkey@0"
I20260806 13:09:07.485674 1914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 13:09:07.486333 1930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:07.486478 1914 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:07.486562 1914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "31ba368ccec941f58fd1aa7e6dcb868c"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "5690a8a5ddcc3ce6f1e1717025b4d691"
server_key_iv: "b35f83cc9a7e46d1ce57bf0fd76d4388"
server_key_version: "encryptionkey@0"
I20260806 13:09:07.486626 1914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:07.510942 1914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:07.512477 1914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:07.512619 1914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:07.516345 1914 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:39465
I20260806 13:09:07.516378 1998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:39465 every 8 connection(s)
I20260806 13:09:07.516680 1914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:09:07.517570 1999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:07.519693 1999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Bootstrap starting.
I20260806 13:09:07.520216 1999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:07.520479 1999 log.cc:824] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:07.521080 1999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: No bootstrap required, opened a new log
I20260806 13:09:07.522634 1999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } }
I20260806 13:09:07.522763 1999 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:07.522797 1999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ba368ccec941f58fd1aa7e6dcb868c, State: Initialized, Role: FOLLOWER
I20260806 13:09:07.522926 1999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } }
I20260806 13:09:07.522985 1999 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:09:07.523008 1999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:09:07.523054 1999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:07.523798 1999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } }
I20260806 13:09:07.523888 1999 leader_election.cc:302] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31ba368ccec941f58fd1aa7e6dcb868c; no voters:
I20260806 13:09:07.524045 1999 leader_election.cc:288] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:09:07.524091 2003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:09:07.524245 2003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 1 LEADER]: Becoming Leader. State: Replica: 31ba368ccec941f58fd1aa7e6dcb868c, State: Running, Role: LEADER
I20260806 13:09:07.524281 1999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:09:07.524340 2003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } }
I20260806 13:09:07.524716 2004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } } }
I20260806 13:09:07.524729 2005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31ba368ccec941f58fd1aa7e6dcb868c. Latest consensus state: current_term: 1 leader_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } } }
I20260806 13:09:07.524936 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 1914
I20260806 13:09:07.524964 2005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:07.525044 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:09:07.524883 2004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:07.525277 14209 external_mini_cluster.cc:1428] Setting key 7cba828ff7e616ccdbcb5b5a0f9efcbb
I20260806 13:09:07.525331 2013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:09:07.525839 2013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:09:07.527535 2013 catalog_manager.cc:1347] Generated new cluster ID: afb5e99d736441549bf5a5692a22c0b4
I20260806 13:09:07.527635 2013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:09:07 dist-test-slave-q9h9 krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:07.533440 2013 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:09:07.533970 2013 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:09:07.536303 2013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Generated new TSK 0
I20260806 13:09:07.536453 2013 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:09:07.537077 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:07.525903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.13.224.126:39465)
0806 13:09:07.526090 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:07.526099 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:07.526133 (+ 34us) server_negotiation.cc:407] Connection header received
0806 13:09:07.526410 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:07.526418 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:07.526481 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:07.526954 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:07.527711 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.529079 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.530118 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.530753 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.530864 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:07.533499 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:07.533504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:07.533504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:07.533516 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:07.535480 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:07.535857 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:07.535859 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:07.535860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:07.535964 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:07.536180 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:07.536181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:07.536181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:07.536445 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:07.536566 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:07.536810 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 13:09:07.536921 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:09:07.560199 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:39465
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:07.642858 2040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:07.643043 2040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:07.643074 2040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:07.644172 2040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:07.644213 2040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:07.644256 2040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:09:07.645691 2040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:39465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:07.645917 2040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:07.646155 2040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:07.651635 2054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:07.651678 2051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:07.651664 2050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:07.651870 2040 server_base.cc:1034] running on GCE node
I20260806 13:09:07.652097 2040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:07.652313 2040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:07.653451 2040 hybrid_clock.cc:648] HybridClock initialized: now 1786021747653438 us; error 31 us; skew 500 ppm
Aug 06 13:09:07 dist-test-slave-q9h9 krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021747, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:07.655827 2040 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:09:07.656205 2040 webserver.cc:460] Webserver started at http://127.13.224.65:38421/ using document root <none> and password file <none>
I20260806 13:09:07.656419 2040 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:07.656467 2040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:07.656591 2040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:09:07.657544 2040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "39953b23bae048ebbbc20d4e90795dec"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "6c0230f41d66623c11c6df0dc57c5802"
server_key_iv: "37e9fad23f92ac32e4278bb3b5f9cf27"
server_key_version: "encryptionkey@0"
I20260806 13:09:07.657866 2040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "39953b23bae048ebbbc20d4e90795dec"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "6c0230f41d66623c11c6df0dc57c5802"
server_key_iv: "37e9fad23f92ac32e4278bb3b5f9cf27"
server_key_version: "encryptionkey@0"
I20260806 13:09:07.659083 2040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:09:07.659714 2064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:07.659858 2040 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:09:07.659938 2040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "39953b23bae048ebbbc20d4e90795dec"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "6c0230f41d66623c11c6df0dc57c5802"
server_key_iv: "37e9fad23f92ac32e4278bb3b5f9cf27"
server_key_version: "encryptionkey@0"
I20260806 13:09:07.660001 2040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:07.679872 2040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:07.681885 2040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:07.682065 2040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:07.682274 2040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:07.682554 2040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:09:07.682586 2040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:07.682614 2040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:09:07.682637 2040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:07.688194 2040 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:43559
I20260806 13:09:07.688264 2222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:43559 every 8 connection(s)
I20260806 13:09:07.688532 2040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
Aug 06 13:09:07 dist-test-slave-q9h9 krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021747, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:07.694769 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 2040
I20260806 13:09:07.694869 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:09:07.695046 14209 external_mini_cluster.cc:1428] Setting key 46281ade374c48163becf527ef567228
I20260806 13:09:07.696942 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:07.689157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:39101 (local address 127.13.224.126:39465)
0806 13:09:07.689207 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:07.689209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:07.689458 (+ 249us) server_negotiation.cc:407] Connection header received
0806 13:09:07.689924 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:07.689925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:07.689935 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:07.689987 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:07.690875 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.691363 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.692252 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.692437 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.692478 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:07.694390 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:07.694391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:07.694391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:07.694398 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:07.695791 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:07.696134 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:07.696135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:07.696135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:07.696196 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:07.696380 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:07.696380 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:07.696381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:07.696504 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:07.696572 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:07.696775 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 13:09:07.696860 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:09:07.697021 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:07.689166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:39101)
0806 13:09:07.689348 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:07.689387 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:07.689847 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:07.690080 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:07.690084 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:07.690620 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:07.690809 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:07.690813 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.691427 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.691427 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:07.692192 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:07.692194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.692487 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.692487 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:07.692579 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:07.692984 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:07.692991 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:07.694300 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:07.695859 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:07.695866 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:07.695867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:07.696068 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:07.696251 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:07.696252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:07.696252 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:07.696333 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:07.696559 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:07.696562 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:07.696664 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 13:09:07.696744 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 13:09:07.696858 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260806 13:09:07.697476 2223 heartbeater.cc:347] Connected to a master server at 127.13.224.126:39465
I20260806 13:09:07.697570 2223 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:07.697736 2223 heartbeater.cc:511] Master 127.13.224.126:39465 requested a full tablet report, sending...
I20260806 13:09:07.698256 1950 ts_manager.cc:194] Registered new tserver with Master: 39953b23bae048ebbbc20d4e90795dec (127.13.224.65:43559)
I20260806 13:09:07.698827 1950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:39101
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:09:07.717535 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:39465
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:07.798153 2231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:07.798352 2231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:07.798375 2231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:07.799465 2231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:07.799500 2231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:07.799525 2231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:09:07.800935 2231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:39465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:07.801127 2231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:07.801316 2231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:07.805946 2239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:07.805917 2240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:07.806095 2231 server_base.cc:1034] running on GCE node
W20260806 13:09:07.806160 2242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:07.806370 2231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:07.806607 2231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:07.807754 2231 hybrid_clock.cc:648] HybridClock initialized: now 1786021747807740 us; error 28 us; skew 500 ppm
Aug 06 13:09:07 dist-test-slave-q9h9 krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021747, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:07.810220 2231 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:09:07.810622 2231 webserver.cc:460] Webserver started at http://127.13.224.66:33775/ using document root <none> and password file <none>
I20260806 13:09:07.810834 2231 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:07.810884 2231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:07.811010 2231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:09:07.812007 2231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "bcb9edc7a2c940f79497ea0296dc048c"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "37e6c1ede2574979f574cd8faf5c8889"
server_key_iv: "b0a5fd1a3c2e1c57380723112f84687e"
server_key_version: "encryptionkey@0"
I20260806 13:09:07.812340 2231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "bcb9edc7a2c940f79497ea0296dc048c"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "37e6c1ede2574979f574cd8faf5c8889"
server_key_iv: "b0a5fd1a3c2e1c57380723112f84687e"
server_key_version: "encryptionkey@0"
I20260806 13:09:07.813513 2231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:09:07.814169 2253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:07.814324 2231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:09:07.814406 2231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "bcb9edc7a2c940f79497ea0296dc048c"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "37e6c1ede2574979f574cd8faf5c8889"
server_key_iv: "b0a5fd1a3c2e1c57380723112f84687e"
server_key_version: "encryptionkey@0"
I20260806 13:09:07.814471 2231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:07.841264 2231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:07.843104 2231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:07.843211 2231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:07.843379 2231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:07.843616 2231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:09:07.843644 2231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:07.843670 2231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:09:07.843685 2231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:07.849038 2231 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:38163
I20260806 13:09:07.849092 2408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:38163 every 8 connection(s)
I20260806 13:09:07.849375 2231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:09:07.852319 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 2231
I20260806 13:09:07.852416 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:09:07.852622 14209 external_mini_cluster.cc:1428] Setting key 1dccebc7c87d6353df5ee7a58576a2a3
Aug 06 13:09:07 dist-test-slave-q9h9 krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021747, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:07.858196 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:07.850007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:43247 (local address 127.13.224.126:39465)
0806 13:09:07.850075 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:07.850077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:07.850259 (+ 182us) server_negotiation.cc:407] Connection header received
0806 13:09:07.850706 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:07.850707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:07.850716 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:07.850773 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:07.851601 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.852084 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.853347 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.853534 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.853583 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:07.855663 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:07.855665 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:07.855666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:07.855673 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:07.856961 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:07.857308 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:07.857309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:07.857310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:07.857379 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:07.857580 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:07.857581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:07.857582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:07.857728 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:07.857796 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:07.858032 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 13:09:07.858107 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 13:09:07.858289 2412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:07.849960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:43247)
0806 13:09:07.850135 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:07.850183 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:07.850630 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:07.850824 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:07.850827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:07.851356 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:07.851536 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:07.851541 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.852185 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.852186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:07.853276 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:07.853278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.853607 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.853608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:07.853734 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:07.854194 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:07.854204 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:07.855572 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:07.857033 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:07.857039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:07.857041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:07.857241 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:07.857441 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:07.857442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:07.857442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:07.857525 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:07.857786 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:07.857792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:07.857898 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 13:09:07.857977 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 13:09:07.858128 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":106,"threads_started":1}
I20260806 13:09:07.858726 2410 heartbeater.cc:347] Connected to a master server at 127.13.224.126:39465
I20260806 13:09:07.858814 2410 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:07.858973 2410 heartbeater.cc:511] Master 127.13.224.126:39465 requested a full tablet report, sending...
I20260806 13:09:07.859338 1950 ts_manager.cc:194] Registered new tserver with Master: bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163)
I20260806 13:09:07.859843 1950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:43247
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:09:07.876314 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:39465
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:07.954883 2420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:07.955058 2420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:07.955089 2420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:07.956182 2420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:07.956223 2420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:07.956256 2420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:09:07.957650 2420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:39465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:07.957870 2420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:07.958088 2420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:07.962607 2431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:07.962626 2433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:07.962598 2430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:07.962821 2420 server_base.cc:1034] running on GCE node
I20260806 13:09:07.963004 2420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:07.963212 2420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:07.964339 2420 hybrid_clock.cc:648] HybridClock initialized: now 1786021747964327 us; error 29 us; skew 500 ppm
Aug 06 13:09:07 dist-test-slave-q9h9 krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021747, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:07.966508 2420 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:09:07.966912 2420 webserver.cc:460] Webserver started at http://127.13.224.67:41467/ using document root <none> and password file <none>
I20260806 13:09:07.967113 2420 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:07.967159 2420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:07.967283 2420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:09:07.968214 2420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "745b15c1468b4eb7ac30b2a8b80d9c70"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "ce674a6703f353a6df41b87be4fca416"
server_key_iv: "95637288791549b8c733fe6e3ce6d058"
server_key_version: "encryptionkey@0"
I20260806 13:09:07.968611 2420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "745b15c1468b4eb7ac30b2a8b80d9c70"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "ce674a6703f353a6df41b87be4fca416"
server_key_iv: "95637288791549b8c733fe6e3ce6d058"
server_key_version: "encryptionkey@0"
I20260806 13:09:07.969794 2420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 13:09:07.970579 2440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:07.970782 2420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 13:09:07.970872 2420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "745b15c1468b4eb7ac30b2a8b80d9c70"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "ce674a6703f353a6df41b87be4fca416"
server_key_iv: "95637288791549b8c733fe6e3ce6d058"
server_key_version: "encryptionkey@0"
I20260806 13:09:07.970937 2420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:07.981976 2420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:07.983604 2420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:07.983718 2420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:07.983906 2420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:07.984189 2420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:09:07.984230 2420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:07.984257 2420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:09:07.984282 2420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:07.989627 2420 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:33975
I20260806 13:09:07.989699 2602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:33975 every 8 connection(s)
I20260806 13:09:07.990034 2420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:09:07.990335 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 2420
I20260806 13:09:07.990437 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:09:07.990619 14209 external_mini_cluster.cc:1428] Setting key e44d604d29d9798cf56b9251ced68e3c
Aug 06 13:09:07 dist-test-slave-q9h9 krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021747, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:07.998164 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:07.991027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:35151 (local address 127.13.224.126:39465)
0806 13:09:07.991075 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:07.991077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:07.991187 (+ 110us) server_negotiation.cc:407] Connection header received
0806 13:09:07.991731 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:07.991732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:07.991741 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:07.991785 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:07.992585 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.993056 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.994001 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.994256 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.994306 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:07.996127 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:07.996129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:07.996129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:07.996135 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:07.997169 (+ 1034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:07.997485 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:07.997486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:07.997486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:07.997529 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:07.997678 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:07.997678 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:07.997678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:07.997767 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:07.997808 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:07.998004 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 13:09:07.998103 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:09:07.998314 2606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:07.990754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:35151)
0806 13:09:07.991071 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:07.991113 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:07.991653 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:07.991836 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:07.991839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:07.992344 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:07.992521 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:07.992526 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.993120 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.993121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:07.993926 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:07.993930 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:07.994324 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:07.994324 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:07.994427 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:07.994808 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:07.994815 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:07.996050 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:07.997230 (+ 1180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:07.997236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:07.997238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:07.997432 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:07.997563 (+ 131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:07.997563 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:07.997564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:07.997641 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:07.997811 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:07.997814 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:07.997913 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 13:09:07.997987 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 13:09:07.998137 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":86,"threads_started":1}
I20260806 13:09:07.998771 2603 heartbeater.cc:347] Connected to a master server at 127.13.224.126:39465
I20260806 13:09:07.998848 2603 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:07.998977 2603 heartbeater.cc:511] Master 127.13.224.126:39465 requested a full tablet report, sending...
I20260806 13:09:07.999290 1950 ts_manager.cc:194] Registered new tserver with Master: 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975)
I20260806 13:09:07.999644 1950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:35151
I20260806 13:09:08.002120 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:09:08.008047 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.003355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47944 (local address 127.13.224.126:39465)
0806 13:09:08.003413 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.003414 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.003493 (+ 79us) server_negotiation.cc:407] Connection header received
0806 13:09:08.003531 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.003531 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.003539 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.003580 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:08.004047 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.004469 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.005092 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.005252 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.005288 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.006277 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:08.006278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:08.006279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:08.006283 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:08.007280 (+ 997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:08.007540 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:08.007541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:08.007541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:08.007582 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:08.007698 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:08.007698 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:08.007699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:08.007806 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:08.007871 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.007940 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.007991 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 13:09:08.009388 1950 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:09:08.010929 1950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:09:08.016546 2622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.012866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:33975 (local address 127.0.0.1:44758)
0806 13:09:08.013181 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.013191 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:08.013241 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:08.013448 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:08.013449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:08.013451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:08.013505 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.013506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.014415 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.014416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.015512 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.015515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.016283 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.016284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.016357 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.016371 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 13:09:08.016428 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 13:09:08.016478 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":62,"threads_started":1}
I20260806 13:09:08.016601 2623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.013043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44758 (local address 127.13.224.67:33975)
0806 13:09:08.013183 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.013187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.013232 (+ 45us) server_negotiation.cc:407] Connection header received
0806 13:09:08.013298 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.013300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.013315 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.013385 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:08.013576 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.014330 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.015616 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.016211 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.016270 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.016324 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.016447 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.016521 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260806 13:09:08.017005 2617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.012681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:38163 (local address 127.0.0.1:48458)
0806 13:09:08.012831 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.012869 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:08.012932 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:08.013238 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:08.013240 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:08.013242 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:08.013300 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.013302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.014837 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.014838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.015900 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.015902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.016755 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.016756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.016851 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.016856 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:09:08.016886 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 13:09:08.016922 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":23,"thread_start_us":51,"threads_started":1}
I20260806 13:09:08.017012 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.012696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:43559 (local address 127.0.0.1:37372)
0806 13:09:08.012985 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.012995 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:08.013043 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:08.013529 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:08.013530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:08.013531 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:08.013558 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.013559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.014882 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.014883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.015739 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.015742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.016559 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.016561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.016890 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.016894 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:09:08.016922 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 13:09:08.016949 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
I20260806 13:09:08.017205 2511 tablet_service.cc:1467] Processing CreateTablet for tablet a86007fbd61b4c15ad88e21abf1154f4 (DEFAULT_TABLE table=test-table [id=593f6370ab7a40abb65b392be25c6a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:08.017449 2620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.012799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48458 (local address 127.13.224.66:38163)
0806 13:09:08.013068 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.013072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.013082 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:09:08.013122 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.013124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.013139 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.013222 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:08.013376 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.014727 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.015977 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.016672 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.017223 (+ 551us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.017301 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.017327 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.017368 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":98,"threads_started":1}
I20260806 13:09:08.017478 2511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a86007fbd61b4c15ad88e21abf1154f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:08.017642 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.012841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37372 (local address 127.13.224.65:43559)
0806 13:09:08.013362 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.013366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.013376 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:09:08.013411 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.013413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.013429 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.013530 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:08.013685 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.014787 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.015808 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.016459 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.017272 (+ 813us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.017375 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.017397 (+ 22us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.017432 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":75,"threads_started":1}
I20260806 13:09:08.018144 2313 tablet_service.cc:1467] Processing CreateTablet for tablet a86007fbd61b4c15ad88e21abf1154f4 (DEFAULT_TABLE table=test-table [id=593f6370ab7a40abb65b392be25c6a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:08.018369 2129 tablet_service.cc:1467] Processing CreateTablet for tablet a86007fbd61b4c15ad88e21abf1154f4 (DEFAULT_TABLE table=test-table [id=593f6370ab7a40abb65b392be25c6a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:08.018476 2313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a86007fbd61b4c15ad88e21abf1154f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:08.018658 2129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a86007fbd61b4c15ad88e21abf1154f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:08.021093 2624 tablet_bootstrap.cc:492] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70: Bootstrap starting.
I20260806 13:09:08.021525 2628 tablet_bootstrap.cc:492] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c: Bootstrap starting.
I20260806 13:09:08.021710 2624 tablet_bootstrap.cc:654] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:08.022166 2627 tablet_bootstrap.cc:492] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Bootstrap starting.
I20260806 13:09:08.022178 2628 tablet_bootstrap.cc:654] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:08.022346 2624 log.cc:824] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:08.022737 2628 log.cc:824] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:08.022748 2627 tablet_bootstrap.cc:654] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:08.023159 2624 tablet_bootstrap.cc:492] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70: No bootstrap required, opened a new log
I20260806 13:09:08.023231 2624 ts_tablet_manager.cc:1397] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:09:08.023331 2627 log.cc:824] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:08.023450 2628 tablet_bootstrap.cc:492] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c: No bootstrap required, opened a new log
I20260806 13:09:08.023514 2628 ts_tablet_manager.cc:1397] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:09:08.025326 2624 raft_consensus.cc:348] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.025516 2624 raft_consensus.cc:374] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:08.025429 2628 raft_consensus.cc:348] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.025559 2624 raft_consensus.cc:729] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 745b15c1468b4eb7ac30b2a8b80d9c70, State: Initialized, Role: FOLLOWER
I20260806 13:09:08.025580 2628 raft_consensus.cc:374] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:08.025619 2628 raft_consensus.cc:729] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb9edc7a2c940f79497ea0296dc048c, State: Initialized, Role: FOLLOWER
I20260806 13:09:08.025679 2624 consensus_queue.cc:260] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.025710 2628 consensus_queue.cc:260] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.025964 2624 ts_tablet_manager.cc:1428] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:09:08.025985 2603 heartbeater.cc:503] Master 127.13.224.126:39465 was elected leader, sending a full tablet report...
I20260806 13:09:08.025964 2410 heartbeater.cc:503] Master 127.13.224.126:39465 was elected leader, sending a full tablet report...
I20260806 13:09:08.025964 2628 ts_tablet_manager.cc:1428] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 13:09:08.026557 2627 tablet_bootstrap.cc:492] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: No bootstrap required, opened a new log
I20260806 13:09:08.026613 2627 ts_tablet_manager.cc:1397] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 13:09:08.028056 2627 raft_consensus.cc:348] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.028163 2627 raft_consensus.cc:374] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:08.028182 2627 raft_consensus.cc:729] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39953b23bae048ebbbc20d4e90795dec, State: Initialized, Role: FOLLOWER
I20260806 13:09:08.028275 2627 consensus_queue.cc:260] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.028501 2627 ts_tablet_manager.cc:1428] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:09:08.028549 2223 heartbeater.cc:503] Master 127.13.224.126:39465 was elected leader, sending a full tablet report...
W20260806 13:09:08.099920 2411 tablet.cc:2369] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:09:08.135773 2635 raft_consensus.cc:482] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:09:08.135896 2635 raft_consensus.cc:504] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.136240 2635 leader_election.cc:288] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163), 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975)
I20260806 13:09:08.139969 2623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.136582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:50665 (local address 127.13.224.67:33975)
0806 13:09:08.136627 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.136628 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.136741 (+ 113us) server_negotiation.cc:407] Connection header received
0806 13:09:08.136790 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.136791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.136798 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.136843 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:08.136962 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.137648 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.139048 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.139628 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.139699 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.139708 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.139859 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.139896 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 13:09:08.139969 2620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.136488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:40823 (local address 127.13.224.66:38163)
0806 13:09:08.136538 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.136540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.136550 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:09:08.136617 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.136618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.136630 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.136684 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:08.136860 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.137598 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.139042 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.139624 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.139700 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.139708 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.139859 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.139896 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 13:09:08.139989 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.136407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:38163 (local address 127.13.224.65:40823)
0806 13:09:08.136468 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.136482 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:08.136544 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:08.136695 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:08.136695 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:08.136697 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:08.136780 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.136783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.137681 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.137682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.138960 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.138964 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.139701 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.139702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.139793 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.139795 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:09:08.139847 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 13:09:08.139894 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":56}
I20260806 13:09:08.139995 2636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.136499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:33975 (local address 127.13.224.65:50665)
0806 13:09:08.136686 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.136695 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:08.136743 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:08.136852 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:08.136852 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:08.136854 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:08.136909 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.136910 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.137721 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.137721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.138961 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.138964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.139701 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.139702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.139789 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.139793 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:09:08.139846 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 13:09:08.139894 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":8,"thread_start_us":98,"threads_started":1}
I20260806 13:09:08.140434 2345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a86007fbd61b4c15ad88e21abf1154f4" candidate_uuid: "39953b23bae048ebbbc20d4e90795dec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb9edc7a2c940f79497ea0296dc048c" is_pre_election: true
I20260806 13:09:08.140519 2540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a86007fbd61b4c15ad88e21abf1154f4" candidate_uuid: "39953b23bae048ebbbc20d4e90795dec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" is_pre_election: true
I20260806 13:09:08.140615 2345 raft_consensus.cc:2454] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39953b23bae048ebbbc20d4e90795dec in term 0.
I20260806 13:09:08.140647 2540 raft_consensus.cc:2454] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39953b23bae048ebbbc20d4e90795dec in term 0.
I20260806 13:09:08.140847 2066 leader_election.cc:302] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39953b23bae048ebbbc20d4e90795dec, bcb9edc7a2c940f79497ea0296dc048c; no voters:
I20260806 13:09:08.140972 2635 raft_consensus.cc:2781] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:09:08.141031 2635 raft_consensus.cc:482] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:09:08.141058 2635 raft_consensus.cc:3037] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:08.141741 2635 raft_consensus.cc:504] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.141846 2635 leader_election.cc:288] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 1 election: Requested vote from peers bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163), 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975)
I20260806 13:09:08.142004 2345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a86007fbd61b4c15ad88e21abf1154f4" candidate_uuid: "39953b23bae048ebbbc20d4e90795dec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb9edc7a2c940f79497ea0296dc048c"
I20260806 13:09:08.142071 2540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a86007fbd61b4c15ad88e21abf1154f4" candidate_uuid: "39953b23bae048ebbbc20d4e90795dec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70"
I20260806 13:09:08.142099 2345 raft_consensus.cc:3037] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:08.142160 2540 raft_consensus.cc:3037] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:08.142802 2345 raft_consensus.cc:2454] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39953b23bae048ebbbc20d4e90795dec in term 1.
I20260806 13:09:08.142889 2540 raft_consensus.cc:2454] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39953b23bae048ebbbc20d4e90795dec in term 1.
I20260806 13:09:08.142956 2066 leader_election.cc:302] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39953b23bae048ebbbc20d4e90795dec, bcb9edc7a2c940f79497ea0296dc048c; no voters:
I20260806 13:09:08.143035 2635 raft_consensus.cc:2781] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:09:08.143186 2635 raft_consensus.cc:686] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 1 LEADER]: Becoming Leader. State: Replica: 39953b23bae048ebbbc20d4e90795dec, State: Running, Role: LEADER
I20260806 13:09:08.143276 2635 consensus_queue.cc:237] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.144089 1950 catalog_manager.cc:5495] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec reported cstate change: term changed from 0 to 1, leader changed from <none> to 39953b23bae048ebbbc20d4e90795dec (127.13.224.65). New cstate: current_term: 1 leader_uuid: "39953b23bae048ebbbc20d4e90795dec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } health_report { overall_health: HEALTHY } } }
I20260806 13:09:08.158962 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.156760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37384 (local address 127.13.224.65:43559)
0806 13:09:08.156807 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.156809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.156817 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:09:08.156855 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.156856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.156876 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.156939 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:09:08.157056 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.157514 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.158249 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.158455 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.158509 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.158656 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:09:08.158766 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:09:08.158834 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.158854 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.158874 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 13:09:08.160740 2540 raft_consensus.cc:1261] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 1 FOLLOWER]: Refusing update from remote peer 39953b23bae048ebbbc20d4e90795dec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:09:08.160740 2345 raft_consensus.cc:1261] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c [term 1 FOLLOWER]: Refusing update from remote peer 39953b23bae048ebbbc20d4e90795dec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:09:08.161077 2637 consensus_queue.cc:1035] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:09:08.161195 2635 consensus_queue.cc:1035] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [LEADER]: Connected to new peer: Peer: permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:09:08.165856 1950 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:09:08.165949 1950 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:09:08.166657 1950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Sending DeleteTablet for 3 replicas of tablet a86007fbd61b4c15ad88e21abf1154f4
I20260806 13:09:08.167001 2313 tablet_service.cc:1514] Processing DeleteTablet for tablet a86007fbd61b4c15ad88e21abf1154f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:08 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:48458
I20260806 13:09:08.167006 2511 tablet_service.cc:1514] Processing DeleteTablet for tablet a86007fbd61b4c15ad88e21abf1154f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:08 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:44758
I20260806 13:09:08.167217 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 2040
I20260806 13:09:08.167275 2129 tablet_service.cc:1514] Processing DeleteTablet for tablet a86007fbd61b4c15ad88e21abf1154f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:08 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:37372
I20260806 13:09:08.178191 2657 tablet_replica.cc:330] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c: stopping tablet replica
I20260806 13:09:08.178308 2657 raft_consensus.cc:2229] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:09:08.178417 2657 raft_consensus.cc:2258] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:09:08.178586 2656 tablet_replica.cc:330] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70: stopping tablet replica
I20260806 13:09:08.178690 2656 raft_consensus.cc:2229] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:09:08.178788 2656 raft_consensus.cc:2258] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:09:08.178884 2657 ts_tablet_manager.cc:1905] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:09:08.179162 2656 ts_tablet_manager.cc:1905] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:09:08.180483 2656 ts_tablet_manager.cc:1918] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:09:08.180534 2657 ts_tablet_manager.cc:1918] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:09:08.180552 2656 log.cc:1196] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/a86007fbd61b4c15ad88e21abf1154f4
I20260806 13:09:08.180603 2657 log.cc:1196] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/a86007fbd61b4c15ad88e21abf1154f4
I20260806 13:09:08.180811 2656 ts_tablet_manager.cc:1939] T a86007fbd61b4c15ad88e21abf1154f4 P 745b15c1468b4eb7ac30b2a8b80d9c70: Deleting consensus metadata
I20260806 13:09:08.180879 2657 ts_tablet_manager.cc:1939] T a86007fbd61b4c15ad88e21abf1154f4 P bcb9edc7a2c940f79497ea0296dc048c: Deleting consensus metadata
I20260806 13:09:08.181248 1932 catalog_manager.cc:4841] TS bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163): tablet a86007fbd61b4c15ad88e21abf1154f4 (table test-table [id=593f6370ab7a40abb65b392be25c6a2d]) successfully deleted
I20260806 13:09:08.181344 1932 catalog_manager.cc:4841] TS 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975): tablet a86007fbd61b4c15ad88e21abf1154f4 (table test-table [id=593f6370ab7a40abb65b392be25c6a2d]) successfully deleted
W20260806 13:09:08.182581 1936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:43559 (error 108)
I20260806 13:09:08.182814 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 2231
I20260806 13:09:08.183025 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.182916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:43559 (local address 127.0.0.1:37388)
0806 13:09:08.182963 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.182996 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:43559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 13:09:08.190399 1936 catalog_manager.cc:4584] TS 39953b23bae048ebbbc20d4e90795dec (127.13.224.65:43559): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a86007fbd61b4c15ad88e21abf1154f4: Network error: Client connection negotiation failed: client connection to 127.13.224.65:43559: connect: Connection refused (error 111)
I20260806 13:09:08.195302 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 2420
I20260806 13:09:08.201166 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 1914
I20260806 13:09:08.211505 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:39465
--webserver_interface=127.13.224.126
--webserver_port=41351
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:39465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:08.290829 2661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:08.291016 2661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:08.291046 2661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:08.292131 2661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:09:08.292168 2661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:08.292187 2661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:08.292203 2661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:09:08.292219 2661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:09:08.293682 2661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:39465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:39465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=41351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:08.293910 2661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:08.294144 2661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:08.303475 2670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:08.303552 2673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:08.303500 2671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:08.303836 2661 server_base.cc:1034] running on GCE node
I20260806 13:09:08.304035 2661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:08.304301 2661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:08.305430 2661 hybrid_clock.cc:648] HybridClock initialized: now 1786021748305417 us; error 28 us; skew 500 ppm
Aug 06 13:09:08 dist-test-slave-q9h9 krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021748, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:08.307701 2661 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:09:08.308089 2661 webserver.cc:460] Webserver started at http://127.13.224.126:41351/ using document root <none> and password file <none>
I20260806 13:09:08.308265 2661 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:08.308313 2661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:08.309504 2661 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:09:08.310083 2684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:08.310226 2661 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:08.310310 2661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "31ba368ccec941f58fd1aa7e6dcb868c"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "5690a8a5ddcc3ce6f1e1717025b4d691"
server_key_iv: "b35f83cc9a7e46d1ce57bf0fd76d4388"
server_key_version: "encryptionkey@0"
I20260806 13:09:08.310673 2661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:08.334736 2661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:08.336297 2661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:08.336418 2661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:08.340106 2661 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:39465
I20260806 13:09:08.340145 2753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:39465 every 8 connection(s)
I20260806 13:09:08.340483 2661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:09:08.341988 2754 sys_catalog.cc:263] Verifying existing consensus state
I20260806 13:09:08.342650 2754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Bootstrap starting.
I20260806 13:09:08.346117 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 2661
I20260806 13:09:08.346457 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:43559
--local_ip_for_outbound_sockets=127.13.224.65
--tserver_master_addrs=127.13.224.126:39465
--webserver_port=38421
--webserver_interface=127.13.224.65
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:09:08.351145 2754 log.cc:824] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:08.352851 2754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:09:08.353101 2754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Bootstrap complete.
I20260806 13:09:08.355206 2754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } }
I20260806 13:09:08.355422 2754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ba368ccec941f58fd1aa7e6dcb868c, State: Initialized, Role: FOLLOWER
I20260806 13:09:08.355566 2754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } }
I20260806 13:09:08.355621 2754 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:09:08.355672 2754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:09:08.355718 2754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 1 FOLLOWER]: Advancing to term 2
I20260806 13:09:08.356659 2754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } }
I20260806 13:09:08.356760 2754 leader_election.cc:302] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31ba368ccec941f58fd1aa7e6dcb868c; no voters:
I20260806 13:09:08.356910 2754 leader_election.cc:288] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 13:09:08.356956 2763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 2 FOLLOWER]: Leader election won for term 2
I20260806 13:09:08.357100 2763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 2 LEADER]: Becoming Leader. State: Replica: 31ba368ccec941f58fd1aa7e6dcb868c, State: Running, Role: LEADER
I20260806 13:09:08.357152 2754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:09:08.357189 2763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } }
I20260806 13:09:08.357434 2763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31ba368ccec941f58fd1aa7e6dcb868c. Latest consensus state: current_term: 2 leader_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } } }
I20260806 13:09:08.357494 2763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:08.357611 2764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } } }
I20260806 13:09:08.357661 2764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:08.357756 2770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:09:08.358520 2770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:09:08.358742 2770 catalog_manager.cc:1259] Loaded cluster ID: afb5e99d736441549bf5a5692a22c0b4
I20260806 13:09:08.358856 2770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:09:08.359068 2770 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 13:09:08.364018 2770 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 13:09:08.366312 2770 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:09:08.366483 2770 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Loaded TSK: 0
I20260806 13:09:08.366683 2770 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 13:09:08.429716 2758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:08.429900 2758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:08.429921 2758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:08.431032 2758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:08.431071 2758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:08.431095 2758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:09:08.432479 2758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:43559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=38421
--tserver_master_addrs=127.13.224.126:39465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:08.432667 2758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:08.432879 2758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:08.438431 2802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:08.438427 2798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:08.438472 2800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:08.438892 2758 server_base.cc:1034] running on GCE node
I20260806 13:09:08.439075 2758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:08.439306 2758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:08.440474 2758 hybrid_clock.cc:648] HybridClock initialized: now 1786021748440457 us; error 32 us; skew 500 ppm
Aug 06 13:09:08 dist-test-slave-q9h9 krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021748, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:08.442904 2758 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:09:08.443342 2758 webserver.cc:460] Webserver started at http://127.13.224.65:38421/ using document root <none> and password file <none>
I20260806 13:09:08.443543 2758 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:08.443598 2758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:08.444859 2758 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:09:08.445475 2814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:08.445627 2758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:09:08.445698 2758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "39953b23bae048ebbbc20d4e90795dec"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "6c0230f41d66623c11c6df0dc57c5802"
server_key_iv: "37e9fad23f92ac32e4278bb3b5f9cf27"
server_key_version: "encryptionkey@0"
I20260806 13:09:08.446043 2758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:08.452471 2758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:08.454329 2758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:08.454438 2758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:08.454623 2758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:08.455063 2822 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 13:09:08.456352 2758 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 13:09:08.456401 2758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260806 13:09:08.456423 2758 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 13:09:08.457039 2758 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 13:09:08.457082 2758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 13:09:08.457122 2822 tablet_bootstrap.cc:492] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Bootstrap starting.
I20260806 13:09:08.464372 2758 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:43559
I20260806 13:09:08.464812 2758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:09:08.468700 2822 log.cc:824] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:08.469000 2974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:43559 every 8 connection(s)
I20260806 13:09:08.471050 2822 tablet_bootstrap.cc:492] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:09:08.471341 2822 tablet_bootstrap.cc:492] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Bootstrap complete.
I20260806 13:09:08.472036 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 2758
I20260806 13:09:08.472321 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:38163
--local_ip_for_outbound_sockets=127.13.224.66
--tserver_master_addrs=127.13.224.126:39465
--webserver_port=33775
--webserver_interface=127.13.224.66
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 13:09:08 dist-test-slave-q9h9 krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021748, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:08.472923 2822 ts_tablet_manager.cc:1397] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.012s
I20260806 13:09:08.474125 2822 raft_consensus.cc:348] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.474428 2822 raft_consensus.cc:729] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39953b23bae048ebbbc20d4e90795dec, State: Initialized, Role: FOLLOWER
I20260806 13:09:08.474577 2822 consensus_queue.cc:260] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.474862 2822 ts_tablet_manager.cc:1428] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:09:08.476683 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.465515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:55315)
0806 13:09:08.465680 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.465723 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:08.466277 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:08.466741 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:08.466752 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:08.467074 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:08.467310 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.467315 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.468987 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.468988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.470202 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.470204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.471035 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.471037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.471174 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.471708 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:08.471717 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:08.473219 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:08.475250 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:08.475255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:08.475258 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:08.475460 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:08.475683 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:08.475684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:08.475684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:08.475792 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:08.476119 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:08.476126 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:08.476247 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 13:09:08.476333 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 13:09:08.476493 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260806 13:09:08.476683 2980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.465646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:55315 (local address 127.13.224.126:39465)
0806 13:09:08.466151 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.466164 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.466211 (+ 47us) server_negotiation.cc:407] Connection header received
0806 13:09:08.466494 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.466505 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.466564 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.466714 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:08.467432 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.468847 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.470309 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.470923 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.471023 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.473345 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:08.473349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:08.473349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:08.473361 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:08.475176 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:08.475532 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:08.475535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:08.475536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:08.475622 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:08.475848 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:08.475849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:08.475849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:08.476057 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:08.476126 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.476393 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.476507 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":72,"threads_started":1}
I20260806 13:09:08.477649 2976 heartbeater.cc:347] Connected to a master server at 127.13.224.126:39465
I20260806 13:09:08.477753 2976 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:08.477912 2976 heartbeater.cc:511] Master 127.13.224.126:39465 requested a full tablet report, sending...
I20260806 13:09:08.478597 2707 ts_manager.cc:194] Registered new tserver with Master: 39953b23bae048ebbbc20d4e90795dec (127.13.224.65:43559)
I20260806 13:09:08.479545 2707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:55315
I20260806 13:09:08.480458 2976 heartbeater.cc:503] Master 127.13.224.126:39465 was elected leader, sending a full tablet report...
I20260806 13:09:08.485673 2984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.479118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:43559 (local address 127.0.0.1:37404)
0806 13:09:08.479256 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.479303 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:08.479365 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:08.482306 (+ 2941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:08.482310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:08.482312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:08.482364 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.482366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.483487 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.483488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.484604 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.484608 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.485465 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.485466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.485523 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.485533 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 13:09:08.485587 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 13:09:08.485622 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260806 13:09:08.485698 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.479289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37404 (local address 127.13.224.65:43559)
0806 13:09:08.482129 (+ 2840us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.482133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.482144 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:09:08.482175 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.482177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.482193 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.482286 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:08.482446 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.483402 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.484694 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.485387 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.485456 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.485504 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.485599 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.485633 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2809,"thread_start_us":65,"threads_started":1}
I20260806 13:09:08.486116 2883 tablet_service.cc:1514] Processing DeleteTablet for tablet a86007fbd61b4c15ad88e21abf1154f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:08 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:37404
I20260806 13:09:08.486639 2986 tablet_replica.cc:330] stopping tablet replica
I20260806 13:09:08.486734 2986 raft_consensus.cc:2229] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:09:08.486795 2986 raft_consensus.cc:2258] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:09:08.487058 2986 ts_tablet_manager.cc:1905] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:09:08.488770 2986 ts_tablet_manager.cc:1918] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:09:08.488834 2986 log.cc:1196] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal/wals/a86007fbd61b4c15ad88e21abf1154f4
I20260806 13:09:08.489033 2986 ts_tablet_manager.cc:1939] T a86007fbd61b4c15ad88e21abf1154f4 P 39953b23bae048ebbbc20d4e90795dec: Deleting consensus metadata
I20260806 13:09:08.489470 2690 catalog_manager.cc:4841] TS 39953b23bae048ebbbc20d4e90795dec (127.13.224.65:43559): tablet a86007fbd61b4c15ad88e21abf1154f4 (table test-table [id=593f6370ab7a40abb65b392be25c6a2d]) successfully deleted
W20260806 13:09:08.555537 2982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:08.555709 2982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:08.555730 2982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:08.556840 2982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:08.556877 2982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:08.556902 2982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:09:08.558378 2982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:38163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=33775
--tserver_master_addrs=127.13.224.126:39465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:08.558552 2982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:08.558712 2982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:08.563156 2994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:08.563165 2996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:08.563234 2993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:08.563366 2982 server_base.cc:1034] running on GCE node
I20260806 13:09:08.563549 2982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:08.563750 2982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:08.564881 2982 hybrid_clock.cc:648] HybridClock initialized: now 1786021748564865 us; error 35 us; skew 500 ppm
Aug 06 13:09:08 dist-test-slave-q9h9 krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021748, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:08.567142 2982 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:09:08.567502 2982 webserver.cc:460] Webserver started at http://127.13.224.66:33775/ using document root <none> and password file <none>
I20260806 13:09:08.567682 2982 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:08.567730 2982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:08.568881 2982 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:09:08.569373 3004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:08.569525 2982 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:08.569600 2982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "bcb9edc7a2c940f79497ea0296dc048c"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "37e6c1ede2574979f574cd8faf5c8889"
server_key_iv: "b0a5fd1a3c2e1c57380723112f84687e"
server_key_version: "encryptionkey@0"
I20260806 13:09:08.569875 2982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:08.584914 2982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:08.586512 2982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:08.586630 2982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:08.586805 2982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:08.587059 2982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:09:08.587091 2982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:08.587117 2982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:09:08.587141 2982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:08.592509 2982 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:38163
I20260806 13:09:08.592569 3172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:38163 every 8 connection(s)
I20260806 13:09:08.592842 2982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
Aug 06 13:09:08 dist-test-slave-q9h9 krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021748, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:08.599790 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 2982
I20260806 13:09:08.600064 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:33975
--local_ip_for_outbound_sockets=127.13.224.67
--tserver_master_addrs=127.13.224.126:39465
--webserver_port=41467
--webserver_interface=127.13.224.67
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:09:08.600817 2980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.593459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:54319 (local address 127.13.224.126:39465)
0806 13:09:08.593511 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.593513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.593683 (+ 170us) server_negotiation.cc:407] Connection header received
0806 13:09:08.594182 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.594184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.594196 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.594265 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:08.595075 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.595549 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.596416 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.596597 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.596636 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.598396 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:08.598398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:08.598398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:08.598405 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:08.599579 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:08.599912 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:08.599913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:08.599913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:08.599979 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:08.600229 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:08.600231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:08.600231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:08.600344 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:08.600378 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.600642 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.600731 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 13:09:08.600849 3176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.593428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:54319)
0806 13:09:08.593576 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.593614 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:08.594092 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:08.594320 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:08.594323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:08.594843 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:08.595010 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.595014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.595607 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.595607 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.596356 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.596358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.596645 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.596645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.596732 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.597089 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:08.597097 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:08.598322 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:08.599640 (+ 1318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:08.599645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:08.599646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:08.599839 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:08.600040 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:08.600040 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:08.600041 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:08.600141 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:08.600413 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:08.600418 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:08.600518 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 13:09:08.600590 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 13:09:08.600698 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260806 13:09:08.601354 3174 heartbeater.cc:347] Connected to a master server at 127.13.224.126:39465
I20260806 13:09:08.601433 3174 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:08.601590 3174 heartbeater.cc:511] Master 127.13.224.126:39465 requested a full tablet report, sending...
I20260806 13:09:08.601991 2707 ts_manager.cc:194] Registered new tserver with Master: bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163)
I20260806 13:09:08.602367 2707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:54319
W20260806 13:09:08.679874 3177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:08.680065 3177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:08.680086 3177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:08.681139 3177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:08.681170 3177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:08.681195 3177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:09:08.682576 3177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:33975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=41467
--tserver_master_addrs=127.13.224.126:39465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:08.682756 3177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:08.682927 3177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:08.687537 3183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:08.687548 3185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:08.687628 3187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:08.687752 3177 server_base.cc:1034] running on GCE node
I20260806 13:09:08.687937 3177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:08.688119 3177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:08.689246 3177 hybrid_clock.cc:648] HybridClock initialized: now 1786021748689224 us; error 38 us; skew 500 ppm
Aug 06 13:09:08 dist-test-slave-q9h9 krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021748, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:08.691495 3177 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:09:08.691918 3177 webserver.cc:460] Webserver started at http://127.13.224.67:41467/ using document root <none> and password file <none>
I20260806 13:09:08.692104 3177 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:08.692164 3177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:08.693414 3177 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:09:08.693919 3198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:08.694080 3177 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:08.694156 3177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "745b15c1468b4eb7ac30b2a8b80d9c70"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "ce674a6703f353a6df41b87be4fca416"
server_key_iv: "95637288791549b8c733fe6e3ce6d058"
server_key_version: "encryptionkey@0"
I20260806 13:09:08.694497 3177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:08.715387 3177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:08.716972 3177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:08.717096 3177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:08.717303 3177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:08.717587 3177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:09:08.717618 3177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:08.717640 3177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:09:08.717662 3177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:08.723440 3177 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:33975
I20260806 13:09:08.723496 3355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:33975 every 8 connection(s)
I20260806 13:09:08.723806 3177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:09:08.725587 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 3177
Aug 06 13:09:08 dist-test-slave-q9h9 krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021748, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:08.731762 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.725968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47946 (local address 127.13.224.126:39465)
0806 13:09:08.726098 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.726100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.726129 (+ 29us) server_negotiation.cc:407] Connection header received
0806 13:09:08.726183 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.726183 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.726194 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.726267 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:08.726763 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.727254 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.727956 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.728158 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.728212 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.729286 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:08.729288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:08.729288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:08.729295 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:08.730737 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:08.731037 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:08.731038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:08.731039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:08.731089 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:08.731259 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:08.731260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:08.731260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:08.731357 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:08.731513 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.731548 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.731667 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
I20260806 13:09:08.732848 2980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.724790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:52935 (local address 127.13.224.126:39465)
0806 13:09:08.724856 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.724859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.724869 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:09:08.725417 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.725418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.725428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.725466 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:08.726587 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.727074 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.728002 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.728199 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.728252 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.730408 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:08.730409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:08.730409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:08.730415 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:08.731657 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:08.732029 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:08.732030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:08.732031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:08.732085 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:08.732317 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:08.732318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:08.732318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:08.732423 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:08.732485 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.732727 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.732792 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":34}
I20260806 13:09:08.733016 3359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.724524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:52935)
0806 13:09:08.724698 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.724745 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:08.725325 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:08.725549 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:08.725553 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:08.726307 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:08.726504 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.726509 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.727152 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.727153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.727925 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.727927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.728273 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.728274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.728385 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.728830 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:08.728838 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:08.730325 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:08.731741 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:08.731749 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:08.731751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:08.731964 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:08.732135 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:08.732135 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:08.732136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:08.732253 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:08.732478 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:08.732484 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:08.732624 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 13:09:08.732707 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 13:09:08.732830 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260806 13:09:08.733392 3356 heartbeater.cc:347] Connected to a master server at 127.13.224.126:39465
I20260806 13:09:08.733476 3356 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:08.733634 3356 heartbeater.cc:511] Master 127.13.224.126:39465 requested a full tablet report, sending...
I20260806 13:09:08.733968 2707 ts_manager.cc:194] Registered new tserver with Master: 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975)
I20260806 13:09:08.734315 2707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:52935
I20260806 13:09:08.743157 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:09:08.745612 2980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.743458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47960 (local address 127.13.224.126:39465)
0806 13:09:08.743502 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.743504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.743594 (+ 90us) server_negotiation.cc:407] Connection header received
0806 13:09:08.743659 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.743660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.743696 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.743757 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:09:08.743853 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.744286 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.744973 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.745148 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.745199 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.745349 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:09:08.745434 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:09:08.745500 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.745535 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.745551 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:09:08.745849 2707 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:09:08.747377 2707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:09:08.749269 2883 tablet_service.cc:1467] Processing CreateTablet for tablet 09a382b0316640ebb8b9601348e23448 (DEFAULT_TABLE table=test-table [id=6c4ea42afb4142bb9a00950883eaaa3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:08.749542 2883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09a382b0316640ebb8b9601348e23448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:08.751178 2822 tablet_bootstrap.cc:492] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec: Bootstrap starting.
I20260806 13:09:08.751513 2822 tablet_bootstrap.cc:654] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:08.752091 2822 tablet_bootstrap.cc:492] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec: No bootstrap required, opened a new log
I20260806 13:09:08.752142 2822 ts_tablet_manager.cc:1397] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 13:09:08.752262 2822 raft_consensus.cc:348] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.752333 2822 raft_consensus.cc:374] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:08.752381 2822 raft_consensus.cc:729] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39953b23bae048ebbbc20d4e90795dec, State: Initialized, Role: FOLLOWER
I20260806 13:09:08.752427 2822 consensus_queue.cc:260] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.752527 2822 ts_tablet_manager.cc:1428] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:08.752725 2984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.749023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:33975 (local address 127.0.0.1:44762)
0806 13:09:08.749239 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.749249 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:08.749288 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:08.749468 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:08.749469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:08.749471 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:08.749506 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.749508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.750588 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.750589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.751330 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.751333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.752525 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.752527 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.752587 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.752591 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:09:08.752623 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 13:09:08.752664 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202}
I20260806 13:09:08.753094 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.749083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.13.224.67:33975)
0806 13:09:08.749271 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.749275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.749285 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:09:08.749354 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.749356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.749371 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.749465 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:08.749595 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.750502 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.751734 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.752426 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.752893 (+ 467us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.752961 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.752987 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.753025 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":84,"threads_started":1}
I20260806 13:09:08.753237 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.749117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:38163 (local address 127.0.0.1:48460)
0806 13:09:08.749510 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.749521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:08.749579 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:08.749785 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:08.749786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:08.749788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:08.749822 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.749823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.750978 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.750979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.751916 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.751920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.753013 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.753014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.753080 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.753084 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:09:08.753129 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 13:09:08.753164 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":86,"threads_started":1}
I20260806 13:09:08.753446 3369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.749222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48460 (local address 127.13.224.66:38163)
0806 13:09:08.749409 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.749412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.749636 (+ 224us) server_negotiation.cc:407] Connection header received
0806 13:09:08.749680 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.749682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.749697 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.749770 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:08.749882 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.750859 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.752014 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.752873 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.752928 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.752980 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.753344 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.753387 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
I20260806 13:09:08.753702 3269 tablet_service.cc:1467] Processing CreateTablet for tablet 09a382b0316640ebb8b9601348e23448 (DEFAULT_TABLE table=test-table [id=6c4ea42afb4142bb9a00950883eaaa3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:08.753993 3269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09a382b0316640ebb8b9601348e23448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:08.753990 3077 tablet_service.cc:1467] Processing CreateTablet for tablet 09a382b0316640ebb8b9601348e23448 (DEFAULT_TABLE table=test-table [id=6c4ea42afb4142bb9a00950883eaaa3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:08.754305 3077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09a382b0316640ebb8b9601348e23448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:08.756613 3371 tablet_bootstrap.cc:492] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70: Bootstrap starting.
I20260806 13:09:08.756910 3372 tablet_bootstrap.cc:492] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c: Bootstrap starting.
I20260806 13:09:08.757439 3371 tablet_bootstrap.cc:654] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:08.757544 3372 tablet_bootstrap.cc:654] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:08.758045 3371 log.cc:824] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:08.758162 3372 log.cc:824] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:08.758824 3371 tablet_bootstrap.cc:492] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70: No bootstrap required, opened a new log
I20260806 13:09:08.758877 3371 ts_tablet_manager.cc:1397] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:09:08.758903 3372 tablet_bootstrap.cc:492] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c: No bootstrap required, opened a new log
I20260806 13:09:08.758971 3372 ts_tablet_manager.cc:1397] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:09:08.760215 3371 raft_consensus.cc:348] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.760339 3371 raft_consensus.cc:374] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:08.760360 3371 raft_consensus.cc:729] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 745b15c1468b4eb7ac30b2a8b80d9c70, State: Initialized, Role: FOLLOWER
I20260806 13:09:08.760456 3371 consensus_queue.cc:260] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.760713 3371 ts_tablet_manager.cc:1428] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:09:08.760736 3356 heartbeater.cc:503] Master 127.13.224.126:39465 was elected leader, sending a full tablet report...
I20260806 13:09:08.760667 3372 raft_consensus.cc:348] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.760803 3372 raft_consensus.cc:374] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:08.760849 3372 raft_consensus.cc:729] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcb9edc7a2c940f79497ea0296dc048c, State: Initialized, Role: FOLLOWER
I20260806 13:09:08.760959 3372 consensus_queue.cc:260] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.761226 3372 ts_tablet_manager.cc:1428] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:09:08.761233 3174 heartbeater.cc:503] Master 127.13.224.126:39465 was elected leader, sending a full tablet report...
W20260806 13:09:08.843362 3175 tablet.cc:2369] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:09:08.856139 2983 raft_consensus.cc:482] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:09:08.856273 2983 raft_consensus.cc:504] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.856611 2983 leader_election.cc:288] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975), bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163)
I20260806 13:09:08.859592 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.856914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:50985 (local address 127.13.224.67:33975)
0806 13:09:08.856980 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.856982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.856992 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:09:08.857023 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.857024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.857034 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.857090 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:08.857212 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.857815 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.858909 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.859293 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.859355 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.859364 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.859490 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.859526 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 13:09:08.859607 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.856831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:33975 (local address 127.13.224.65:50985)
0806 13:09:08.856882 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.856894 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:08.856952 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:08.857097 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:08.857098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:08.857099 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:08.857155 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.857156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.857922 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.857923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.858822 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.858825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.859361 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.859362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.859433 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.859436 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:09:08.859479 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 13:09:08.859517 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 13:09:08.859918 3299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09a382b0316640ebb8b9601348e23448" candidate_uuid: "39953b23bae048ebbbc20d4e90795dec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" is_pre_election: true
I20260806 13:09:08.859975 3379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.856924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:38163 (local address 127.13.224.65:55731)
0806 13:09:08.857111 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.857120 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:08.857170 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:08.857298 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:08.857298 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:08.857300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:08.857678 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.857680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.858375 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.858376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.859129 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:08.859131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.859681 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.859682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:08.859761 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.859774 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 13:09:08.859848 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 13:09:08.859885 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":118,"threads_started":1}
I20260806 13:09:08.859974 3369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.857024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:55731 (local address 127.13.224.66:38163)
0806 13:09:08.857076 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.857078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.857167 (+ 89us) server_negotiation.cc:407] Connection header received
0806 13:09:08.857220 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.857221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.857231 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.857294 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:08.857746 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.858301 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.859212 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.859604 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.859666 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.859674 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.859873 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.859913 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 13:09:08.860075 3299 raft_consensus.cc:2454] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39953b23bae048ebbbc20d4e90795dec in term 0.
I20260806 13:09:08.860311 2816 leader_election.cc:302] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39953b23bae048ebbbc20d4e90795dec, 745b15c1468b4eb7ac30b2a8b80d9c70; no voters:
I20260806 13:09:08.860417 3103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09a382b0316640ebb8b9601348e23448" candidate_uuid: "39953b23bae048ebbbc20d4e90795dec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb9edc7a2c940f79497ea0296dc048c" is_pre_election: true
I20260806 13:09:08.860445 2983 raft_consensus.cc:2781] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:09:08.860512 2983 raft_consensus.cc:482] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:09:08.860541 2983 raft_consensus.cc:3037] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:08.860569 3103 raft_consensus.cc:2454] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39953b23bae048ebbbc20d4e90795dec in term 0.
I20260806 13:09:08.861236 2983 raft_consensus.cc:504] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.861349 2983 leader_election.cc:288] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 1 election: Requested vote from peers 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975), bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163)
I20260806 13:09:08.861518 3299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09a382b0316640ebb8b9601348e23448" candidate_uuid: "39953b23bae048ebbbc20d4e90795dec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70"
I20260806 13:09:08.861543 3103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09a382b0316640ebb8b9601348e23448" candidate_uuid: "39953b23bae048ebbbc20d4e90795dec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcb9edc7a2c940f79497ea0296dc048c"
I20260806 13:09:08.861593 3299 raft_consensus.cc:3037] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:08.861604 3103 raft_consensus.cc:3037] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:08.862280 3103 raft_consensus.cc:2454] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39953b23bae048ebbbc20d4e90795dec in term 1.
I20260806 13:09:08.862407 3299 raft_consensus.cc:2454] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39953b23bae048ebbbc20d4e90795dec in term 1.
I20260806 13:09:08.862435 2816 leader_election.cc:302] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39953b23bae048ebbbc20d4e90795dec, bcb9edc7a2c940f79497ea0296dc048c; no voters:
I20260806 13:09:08.862509 2983 raft_consensus.cc:2781] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:09:08.862658 2983 raft_consensus.cc:686] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 LEADER]: Becoming Leader. State: Replica: 39953b23bae048ebbbc20d4e90795dec, State: Running, Role: LEADER
I20260806 13:09:08.862720 2983 consensus_queue.cc:237] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:08.863472 2706 catalog_manager.cc:5495] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec reported cstate change: term changed from 0 to 1, leader changed from <none> to 39953b23bae048ebbbc20d4e90795dec (127.13.224.65). New cstate: current_term: 1 leader_uuid: "39953b23bae048ebbbc20d4e90795dec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } health_report { overall_health: HEALTHY } } }
I20260806 13:09:08.868037 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.866198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37420 (local address 127.13.224.65:43559)
0806 13:09:08.866233 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:08.866234 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:08.866259 (+ 25us) server_negotiation.cc:407] Connection header received
0806 13:09:08.866326 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:08.866326 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:08.866342 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:08.866387 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:09:08.866462 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.866894 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.867525 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:08.867695 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:08.867739 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:08.867833 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:09:08.867914 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:09:08.867958 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:08.867978 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 13:09:08.867992 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:09:08.869320 3103 raft_consensus.cc:1261] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c [term 1 FOLLOWER]: Refusing update from remote peer 39953b23bae048ebbbc20d4e90795dec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:09:08.869338 3299 raft_consensus.cc:1261] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 1 FOLLOWER]: Refusing update from remote peer 39953b23bae048ebbbc20d4e90795dec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:09:08.869577 2983 consensus_queue.cc:1035] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:09:08.869705 3380 consensus_queue.cc:1035] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [LEADER]: Connected to new peer: Peer: permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:09:08.872011 3384 mvcc.cc:205] Tried to move back new op lower bound from 7315545083366666240 to 7315545083342245888. Current Snapshot: MvccSnapshot[applied={T|T < 7315545083366666240}]
I20260806 13:09:08.873649 3383 mvcc.cc:205] Tried to move back new op lower bound from 7315545083366666240 to 7315545083342245888. Current Snapshot: MvccSnapshot[applied={T|T < 7315545083366666240 or (T in {7315545083366666240})}]
I20260806 13:09:08.873785 2707 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:09:08.873876 2707 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:09:08.874609 2707 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Sending DeleteTablet for 3 replicas of tablet 09a382b0316640ebb8b9601348e23448
I20260806 13:09:08.874897 2883 tablet_service.cc:1514] Processing DeleteTablet for tablet 09a382b0316640ebb8b9601348e23448 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:08 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:37404
I20260806 13:09:08.875003 2986 tablet_replica.cc:330] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec: stopping tablet replica
I20260806 13:09:08.875067 2986 raft_consensus.cc:2229] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:09:08.875099 3077 tablet_service.cc:1514] Processing DeleteTablet for tablet 09a382b0316640ebb8b9601348e23448 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:08 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:48460
I20260806 13:09:08.875144 2986 raft_consensus.cc:2258] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:09:08.875123 3269 tablet_service.cc:1514] Processing DeleteTablet for tablet 09a382b0316640ebb8b9601348e23448 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:08 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:44762
I20260806 13:09:08.875439 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 2758
I20260806 13:09:08.875456 2986 ts_tablet_manager.cc:1905] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:09:08.879860 3394 tablet_replica.cc:330] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c: stopping tablet replica
I20260806 13:09:08.879966 3394 raft_consensus.cc:2229] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:09:08.880051 3394 raft_consensus.cc:2258] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:09:08.881584 3396 tablet_replica.cc:330] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70: stopping tablet replica
I20260806 13:09:08.881664 3396 raft_consensus.cc:2229] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:09:08.881729 3396 raft_consensus.cc:2258] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:09:08.883419 3394 ts_tablet_manager.cc:1905] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:09:08.883436 3396 ts_tablet_manager.cc:1905] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:09:08.884855 3394 ts_tablet_manager.cc:1918] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:09:08.884919 3394 log.cc:1196] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/09a382b0316640ebb8b9601348e23448
I20260806 13:09:08.885145 3394 ts_tablet_manager.cc:1939] T 09a382b0316640ebb8b9601348e23448 P bcb9edc7a2c940f79497ea0296dc048c: Deleting consensus metadata
I20260806 13:09:08.885512 2686 catalog_manager.cc:4841] TS bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163): tablet 09a382b0316640ebb8b9601348e23448 (table test-table [id=6c4ea42afb4142bb9a00950883eaaa3b]) successfully deleted
I20260806 13:09:08.885751 3396 ts_tablet_manager.cc:1918] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:09:08.885807 3396 log.cc:1196] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/09a382b0316640ebb8b9601348e23448
I20260806 13:09:08.886051 3396 ts_tablet_manager.cc:1939] T 09a382b0316640ebb8b9601348e23448 P 745b15c1468b4eb7ac30b2a8b80d9c70: Deleting consensus metadata
I20260806 13:09:08.886410 2686 catalog_manager.cc:4841] TS 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975): tablet 09a382b0316640ebb8b9601348e23448 (table test-table [id=6c4ea42afb4142bb9a00950883eaaa3b]) successfully deleted
W20260806 13:09:08.890558 2690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:43559 (error 108)
I20260806 13:09:08.890941 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 2982
I20260806 13:09:08.891029 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:08.890917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:43559 (local address 127.0.0.1:37432)
0806 13:09:08.890959 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:08.890996 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:43559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260806 13:09:08.891134 2690 catalog_manager.cc:4584] TS 39953b23bae048ebbbc20d4e90795dec (127.13.224.65:43559): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 09a382b0316640ebb8b9601348e23448: Network error: Client connection negotiation failed: client connection to 127.13.224.65:43559: connect: Connection refused (error 111)
I20260806 13:09:08.901110 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 3177
I20260806 13:09:08.915620 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 2661
I20260806 13:09:08.924335 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:39465
--webserver_interface=127.13.224.126
--webserver_port=41351
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:39465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:09.004087 3401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:09.004285 3401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:09.004305 3401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:09.005385 3401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:09:09.005419 3401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:09.005431 3401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:09.005442 3401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:09:09.005456 3401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:09:09.006925 3401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:39465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:39465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=41351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:09.007126 3401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:09.007313 3401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:09.011786 3408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:09.011924 3401 server_base.cc:1034] running on GCE node
W20260806 13:09:09.011937 3412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:09.011803 3407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:09.012269 3401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:09.012563 3401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:09.013700 3401 hybrid_clock.cc:648] HybridClock initialized: now 1786021749013685 us; error 30 us; skew 500 ppm
Aug 06 13:09:09 dist-test-slave-q9h9 krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021749, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:09.015951 3401 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:09:09.016387 3401 webserver.cc:460] Webserver started at http://127.13.224.126:41351/ using document root <none> and password file <none>
I20260806 13:09:09.016587 3401 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:09.016635 3401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:09.017838 3401 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:09:09.018508 3423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:09.018646 3401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:09.018729 3401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "31ba368ccec941f58fd1aa7e6dcb868c"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "5690a8a5ddcc3ce6f1e1717025b4d691"
server_key_iv: "b35f83cc9a7e46d1ce57bf0fd76d4388"
server_key_version: "encryptionkey@0"
I20260806 13:09:09.019027 3401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:09.026907 3401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:09.028398 3401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:09.028517 3401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:09.032177 3401 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:39465
I20260806 13:09:09.032207 3500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:39465 every 8 connection(s)
I20260806 13:09:09.032531 3401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:09:09.033957 3501 sys_catalog.cc:263] Verifying existing consensus state
I20260806 13:09:09.034600 3501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Bootstrap starting.
I20260806 13:09:09.038697 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 3401
I20260806 13:09:09.039052 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:43559
--local_ip_for_outbound_sockets=127.13.224.65
--tserver_master_addrs=127.13.224.126:39465
--webserver_port=38421
--webserver_interface=127.13.224.65
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:09:09.043203 3501 log.cc:824] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:09.045188 3501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:09:09.045452 3501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c: Bootstrap complete.
I20260806 13:09:09.047493 3501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } }
I20260806 13:09:09.047698 3501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ba368ccec941f58fd1aa7e6dcb868c, State: Initialized, Role: FOLLOWER
I20260806 13:09:09.047832 3501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } }
I20260806 13:09:09.047889 3501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:09:09.047916 3501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:09:09.047962 3501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 2 FOLLOWER]: Advancing to term 3
I20260806 13:09:09.048874 3501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } }
I20260806 13:09:09.048959 3501 leader_election.cc:302] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31ba368ccec941f58fd1aa7e6dcb868c; no voters:
I20260806 13:09:09.049100 3501 leader_election.cc:288] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 13:09:09.049146 3505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 3 FOLLOWER]: Leader election won for term 3
I20260806 13:09:09.049264 3505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [term 3 LEADER]: Becoming Leader. State: Replica: 31ba368ccec941f58fd1aa7e6dcb868c, State: Running, Role: LEADER
I20260806 13:09:09.049368 3501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:09:09.049350 3505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } }
I20260806 13:09:09.049741 3509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } } }
I20260806 13:09:09.049836 3509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:09.050040 3510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31ba368ccec941f58fd1aa7e6dcb868c. Latest consensus state: current_term: 3 leader_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31ba368ccec941f58fd1aa7e6dcb868c" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 39465 } } }
I20260806 13:09:09.050319 3510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31ba368ccec941f58fd1aa7e6dcb868c [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:09.050228 3516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:09:09.051151 3516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:09:09.051299 3516 catalog_manager.cc:1259] Loaded cluster ID: afb5e99d736441549bf5a5692a22c0b4
I20260806 13:09:09.051501 3516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 13:09:09.051838 3516 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f26a63179ad google::LogMessage::Fail() at ??:0
@ 0x7f26a631a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f26a63174de google::LogMessage::Flush() at ??:0
@ 0x7f26a6318a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f26ae23c0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f26ae2458b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f26a81e50d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f26a81da954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f26a903e6db start_thread at ??:0
@ 0x7f26a4e1561f clone at ??:0
W20260806 13:09:09.134232 3503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:09.134411 3503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:09.134438 3503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:09.135550 3503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:09.135587 3503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:09.135612 3503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:09:09.137001 3503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:43559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=38421
--tserver_master_addrs=127.13.224.126:39465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:09.137214 3503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:09.137424 3503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:09.142102 3560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:09.142144 3555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:09.142297 3556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:09.142705 3503 server_base.cc:1034] running on GCE node
I20260806 13:09:09.142915 3503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:09.143188 3503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:09.144349 3503 hybrid_clock.cc:648] HybridClock initialized: now 1786021749144329 us; error 32 us; skew 500 ppm
Aug 06 13:09:09 dist-test-slave-q9h9 krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021749, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:09.147414 3503 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:09:09.147954 3503 webserver.cc:460] Webserver started at http://127.13.224.65:38421/ using document root <none> and password file <none>
I20260806 13:09:09.148207 3503 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:09.148267 3503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:09.149865 3503 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:09:09.150743 3568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:09.150907 3503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:09:09.150990 3503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "39953b23bae048ebbbc20d4e90795dec"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "6c0230f41d66623c11c6df0dc57c5802"
server_key_iv: "37e9fad23f92ac32e4278bb3b5f9cf27"
server_key_version: "encryptionkey@0"
I20260806 13:09:09.151286 3503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:09.170046 3503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:09.171990 3503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:09.172106 3503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:09.172305 3503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:09.172742 3577 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 13:09:09.174634 3503 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 13:09:09.174676 3503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 13:09:09.174722 3503 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 13:09:09.175311 3503 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 13:09:09.175343 3503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 13:09:09.175426 3577 tablet_bootstrap.cc:492] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec: Bootstrap starting.
I20260806 13:09:09.177181 3577 log.cc:824] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:09.180110 3577 tablet_bootstrap.cc:492] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:09:09.180405 3577 tablet_bootstrap.cc:492] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec: Bootstrap complete.
I20260806 13:09:09.180555 3577 ts_tablet_manager.cc:1397] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 13:09:09.181914 3503 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:43559
I20260806 13:09:09.182006 3744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:43559 every 8 connection(s)
I20260806 13:09:09.182281 3503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:09:09.182317 3577 raft_consensus.cc:348] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:09.182723 3577 raft_consensus.cc:729] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39953b23bae048ebbbc20d4e90795dec, State: Initialized, Role: FOLLOWER
I20260806 13:09:09.182860 3577 consensus_queue.cc:260] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:09.183110 3577 ts_tablet_manager.cc:1428] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260806 13:09:09.184765 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 3503
I20260806 13:09:09.185020 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:38163
--local_ip_for_outbound_sockets=127.13.224.66
--tserver_master_addrs=127.13.224.126:39465
--webserver_port=33775
--webserver_interface=127.13.224.66
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:09.234892 3749 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:09:09.183313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:56271)
0806 13:09:09.183479 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:09.183521 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:09.183682 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:09.234691 (+ 51009us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
W20260806 13:09:09.235144 3745 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:39465 (0 consecutive failures): Network error: Failed to ping master at 127.13.224.126:39465: Client connection negotiation failed: client connection to 127.13.224.126:39465: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 13:09:09.235476 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:09.235370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:50655)
0806 13:09:09.235416 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:09.235455 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 13:09:09.235764 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:09.235677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:33085)
0806 13:09:09.235714 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:09.235736 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 13:09:09.235869 3745 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 13:09:09.267668 3751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:09.267860 3751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:09.267890 3751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:09.268997 3751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:09.269037 3751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:09.269073 3751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:09:09.270506 3751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:38163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=33775
--tserver_master_addrs=127.13.224.126:39465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:09.270726 3751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:09.270934 3751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:09.275512 3760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:09.275514 3758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:09.275537 3762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:09.275660 3751 server_base.cc:1034] running on GCE node
I20260806 13:09:09.275933 3751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:09.276127 3751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:09.277271 3751 hybrid_clock.cc:648] HybridClock initialized: now 1786021749277257 us; error 34 us; skew 500 ppm
Aug 06 13:09:09 dist-test-slave-q9h9 krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021749, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:09.279754 3751 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:09:09.280139 3751 webserver.cc:460] Webserver started at http://127.13.224.66:33775/ using document root <none> and password file <none>
I20260806 13:09:09.280349 3751 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:09.280395 3751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:09.281596 3751 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:09:09.282137 3772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:09.282272 3751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:09.282351 3751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "bcb9edc7a2c940f79497ea0296dc048c"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "37e6c1ede2574979f574cd8faf5c8889"
server_key_iv: "b0a5fd1a3c2e1c57380723112f84687e"
server_key_version: "encryptionkey@0"
I20260806 13:09:09.282651 3751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:09.294677 3751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:09.296379 3751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:09.296499 3751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:09.296701 3751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:09.296973 3751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:09:09.297005 3751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:09.297026 3751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:09:09.297049 3751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:09.302719 3751 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:38163
I20260806 13:09:09.302803 3943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:38163 every 8 connection(s)
I20260806 13:09:09.303107 3751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:09:09.304100 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:09.303732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:45277)
0806 13:09:09.303878 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:09.303950 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
W20260806 13:09:09.304464 3944 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:39465 (0 consecutive failures): Network error: Failed to ping master at 127.13.224.126:39465: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
I20260806 13:09:09.304709 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:09.304627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:41435)
0806 13:09:09.304665 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:09.304689 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 13:09:09.304942 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:09.304874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:60129)
0806 13:09:09.304905 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:09.304924 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260806 13:09:09.305029 3944 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 13:09:09.310784 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 3751
I20260806 13:09:09.311012 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:33975
--local_ip_for_outbound_sockets=127.13.224.67
--tserver_master_addrs=127.13.224.126:39465
--webserver_port=41467
--webserver_interface=127.13.224.67
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:45101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:09.391311 3949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:09.391506 3949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:09.391528 3949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:09.392627 3949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:09.392670 3949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:09.392694 3949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:09:09.394192 3949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:45101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:33975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=41467
--tserver_master_addrs=127.13.224.126:39465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:09.394428 3949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:09.394651 3949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:09.399500 3956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:09.399507 3958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:09.399507 3960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:09.399712 3949 server_base.cc:1034] running on GCE node
I20260806 13:09:09.399910 3949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:09.400136 3949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:09.401290 3949 hybrid_clock.cc:648] HybridClock initialized: now 1786021749401278 us; error 23 us; skew 500 ppm
Aug 06 13:09:09 dist-test-slave-q9h9 krb5kdc[1897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021749, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:09.403779 3949 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:09:09.404173 3949 webserver.cc:460] Webserver started at http://127.13.224.67:41467/ using document root <none> and password file <none>
I20260806 13:09:09.404371 3949 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:09.404415 3949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:09.405642 3949 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:09:09.406162 3970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:09.406314 3949 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:09.406407 3949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "745b15c1468b4eb7ac30b2a8b80d9c70"
format_stamp: "Formatted at 2026-08-06 13:09:07 on dist-test-slave-q9h9"
server_key: "ce674a6703f353a6df41b87be4fca416"
server_key_iv: "95637288791549b8c733fe6e3ce6d058"
server_key_version: "encryptionkey@0"
I20260806 13:09:09.406723 3949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:09.422830 3949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:09.424475 3949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:09.424602 3949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:09.424803 3949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:09.425081 3949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:09:09.425114 3949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:09.425143 3949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:09:09.425166 3949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:09.430490 3949 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:33975
I20260806 13:09:09.430565 4133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:33975 every 8 connection(s)
I20260806 13:09:09.430852 3949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:09:09.431806 4138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:09.431451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:43511)
0806 13:09:09.431586 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:09.431664 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
W20260806 13:09:09.432181 4135 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:39465 (0 consecutive failures): Network error: Failed to ping master at 127.13.224.126:39465: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
I20260806 13:09:09.432425 4138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:09.432344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:60559)
0806 13:09:09.432382 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:09.432408 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 13:09:09.432703 4138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:09.432621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:33865)
0806 13:09:09.432662 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:09.432684 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 13:09:09.432790 4135 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 13:09:09.436308 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 3949
I20260806 13:09:10.236403 4140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:10.236184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:50295)
0806 13:09:10.236307 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:10.236341 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260806 13:09:10.305473 4141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:10.305282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:52029)
0806 13:09:10.305395 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:10.305426 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260806 13:09:10.433213 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:10.433033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:45893)
0806 13:09:10.433129 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:10.433164 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":55,"threads_started":1}
I20260806 13:09:10.908705 4145 raft_consensus.cc:482] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:09:10.908839 4145 raft_consensus.cc:504] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:10.909109 4145 leader_election.cc:288] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975), bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163)
Aug 06 13:09:10 dist-test-slave-q9h9 krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021749, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.66@KRBTEST.COM
Aug 06 13:09:10 dist-test-slave-q9h9 krb5kdc[1897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021749, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.67@KRBTEST.COM
I20260806 13:09:10.919154 4148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:10.909389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:41549 (local address 127.13.224.67:33975)
0806 13:09:10.909575 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:10.909588 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:10.909628 (+ 40us) server_negotiation.cc:407] Connection header received
0806 13:09:10.910152 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:10.910154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:10.910193 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:10.910359 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:10.911077 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:10.911929 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:10.912857 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:10.913235 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:10.913326 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:10.915949 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:10.915952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:10.915952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:10.915961 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:10.917511 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:10.917850 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:10.917853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:10.917853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:10.917936 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:10.918224 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:10.918225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:10.918226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:10.918519 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:10.918602 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:10.918916 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 13:09:10.919043 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":51,"threads_started":1}
I20260806 13:09:10.919154 4150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:10.909506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:43159 (local address 127.13.224.66:38163)
0806 13:09:10.909647 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:10.909656 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:10.909674 (+ 18us) server_negotiation.cc:407] Connection header received
0806 13:09:10.910145 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:10.910148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:10.910192 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:10.910316 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:10.911210 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:10.912035 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:10.912858 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:10.913228 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:10.913324 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:10.915587 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:10.915590 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:10.915590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:10.915598 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:10.917349 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:10.917810 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:10.917813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:10.917813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:10.917925 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:10.918224 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:10.918225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:10.918226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:10.918519 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:10.918602 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:10.918916 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 13:09:10.919043 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260806 13:09:10.919162 4147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:10.909402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:38163 (local address 127.13.224.65:43159)
0806 13:09:10.909483 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:10.909492 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:10.909540 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:10.910359 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:10.910362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:10.910761 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:10.911105 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:10.911108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:10.912127 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:10.912128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:10.912775 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:10.912777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:10.913305 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:10.913306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:10.913427 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:10.913913 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:10.913920 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:10.915478 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:10.917424 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:10.917431 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:10.917433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:10.917736 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:10.917996 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:10.917996 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:10.917996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:10.918158 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:10.918583 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:10.918588 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:10.918733 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 13:09:10.918856 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 13:09:10.919022 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":9,"thread_start_us":37,"threads_started":1}
I20260806 13:09:10.919461 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:10.909264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:33975 (local address 127.13.224.65:41549)
0806 13:09:10.909360 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:10.909370 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:10.909448 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:10.910359 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:10.910362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:10.910764 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:10.910967 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:10.910973 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:10.912018 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:10.912019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:10.912776 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:10.912779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:10.913312 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:10.913313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:10.913427 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:10.913971 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:10.913976 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:10.915858 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:10.917576 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:10.917577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:10.917577 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:10.917788 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:10.917995 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:10.917995 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:10.917996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:10.918158 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:10.918583 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:10.918588 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:10.918733 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 13:09:10.918856 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 13:09:10.919084 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":110,"thread_start_us":43,"threads_started":1}
W20260806 13:09:10.920056 3569 leader_election.cc:341] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163): Not found: Tablet not found: 09a382b0316640ebb8b9601348e23448
W20260806 13:09:10.920131 3569 leader_election.cc:341] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975): Not found: Tablet not found: 09a382b0316640ebb8b9601348e23448
I20260806 13:09:10.920157 3569 leader_election.cc:302] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39953b23bae048ebbbc20d4e90795dec; no voters: 745b15c1468b4eb7ac30b2a8b80d9c70, bcb9edc7a2c940f79497ea0296dc048c
I20260806 13:09:10.920253 4145 raft_consensus.cc:2726] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:09:11.237003 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:11.236879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:53683)
0806 13:09:11.236928 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:11.236961 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 13:09:11.306100 4151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:11.305877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:47615)
0806 13:09:11.305988 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:11.306056 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260806 13:09:11.433722 4153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:11.433544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:51853)
0806 13:09:11.433650 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:11.433684 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":52,"threads_started":1}
I20260806 13:09:12.237613 4155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:12.237408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:35479)
0806 13:09:12.237519 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:12.237550 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260806 13:09:12.306694 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:12.306484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:37907)
0806 13:09:12.306614 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:12.306652 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
I20260806 13:09:12.434365 4157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:12.434163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:48929)
0806 13:09:12.434283 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:12.434324 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
I20260806 13:09:12.766677 4159 raft_consensus.cc:482] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:09:12.766789 4159 raft_consensus.cc:504] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:12.766938 4159 leader_election.cc:288] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975), bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163)
W20260806 13:09:12.767364 3569 leader_election.cc:341] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975): Not found: Tablet not found: 09a382b0316640ebb8b9601348e23448
W20260806 13:09:12.767436 3569 leader_election.cc:341] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163): Not found: Tablet not found: 09a382b0316640ebb8b9601348e23448
I20260806 13:09:12.767457 3569 leader_election.cc:302] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39953b23bae048ebbbc20d4e90795dec; no voters: 745b15c1468b4eb7ac30b2a8b80d9c70, bcb9edc7a2c940f79497ea0296dc048c
I20260806 13:09:12.767537 4159 raft_consensus.cc:2726] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:09:13.238222 4160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:13.237995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:38383)
0806 13:09:13.238129 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:13.238163 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260806 13:09:13.307292 4161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:13.307111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:34469)
0806 13:09:13.307219 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:13.307254 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":61,"threads_started":1}
I20260806 13:09:13.434952 4162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:13.434738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:35789)
0806 13:09:13.434881 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:13.434914 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260806 13:09:14.238834 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:14.238644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:40327)
0806 13:09:14.238754 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:14.238789 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
I20260806 13:09:14.307826 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:14.307642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:37915)
0806 13:09:14.307752 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:14.307786 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
I20260806 13:09:14.435559 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:14.435355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:48603)
0806 13:09:14.435477 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:14.435516 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260806 13:09:14.573688 4168 raft_consensus.cc:482] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:09:14.573805 4168 raft_consensus.cc:504] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:14.574069 4168 leader_election.cc:288] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975), bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163)
W20260806 13:09:14.574496 3569 leader_election.cc:341] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975): Not found: Tablet not found: 09a382b0316640ebb8b9601348e23448
W20260806 13:09:14.574563 3569 leader_election.cc:341] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163): Not found: Tablet not found: 09a382b0316640ebb8b9601348e23448
I20260806 13:09:14.574579 3569 leader_election.cc:302] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39953b23bae048ebbbc20d4e90795dec; no voters: 745b15c1468b4eb7ac30b2a8b80d9c70, bcb9edc7a2c940f79497ea0296dc048c
I20260806 13:09:14.574642 4168 raft_consensus.cc:2726] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:09:15.239393 4169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:15.239200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:41637)
0806 13:09:15.239312 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:15.239347 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
I20260806 13:09:15.308919 4170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:15.308140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:59019)
0806 13:09:15.308825 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:15.308867 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":632,"threads_started":1}
I20260806 13:09:15.436124 4172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:15.435937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:53905)
0806 13:09:15.436049 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:15.436086 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260806 13:09:16.239980 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:16.239777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:36045)
0806 13:09:16.239899 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:16.239933 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
I20260806 13:09:16.309504 4174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:16.309315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:55427)
0806 13:09:16.309429 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:16.309463 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260806 13:09:16.436794 4175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:16.436559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:36223)
0806 13:09:16.436697 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:16.436747 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
I20260806 13:09:17.240619 4177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:17.240415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:37305)
0806 13:09:17.240538 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:17.240573 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260806 13:09:17.310107 4180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:17.309882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:58499)
0806 13:09:17.309993 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:17.310061 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260806 13:09:17.437403 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:17.437169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:58405)
0806 13:09:17.437324 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:17.437363 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":78,"threads_started":1}
I20260806 13:09:17.641325 4182 raft_consensus.cc:482] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:09:17.641420 4182 raft_consensus.cc:504] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:17.641562 4182 leader_election.cc:288] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975), bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163)
W20260806 13:09:17.641916 3569 leader_election.cc:341] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975): Not found: Tablet not found: 09a382b0316640ebb8b9601348e23448
W20260806 13:09:17.641994 3569 leader_election.cc:341] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163): Not found: Tablet not found: 09a382b0316640ebb8b9601348e23448
I20260806 13:09:17.642037 3569 leader_election.cc:302] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39953b23bae048ebbbc20d4e90795dec; no voters: 745b15c1468b4eb7ac30b2a8b80d9c70, bcb9edc7a2c940f79497ea0296dc048c
I20260806 13:09:17.642143 4182 raft_consensus.cc:2726] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:09:18.241261 4184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:18.241012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:54231)
0806 13:09:18.241177 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:18.241216 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":84,"threads_started":1}
I20260806 13:09:18.310670 4186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:18.310480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:51007)
0806 13:09:18.310596 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:18.310631 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260806 13:09:18.438004 4187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:18.437780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:43095)
0806 13:09:18.437932 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:18.437966 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260806 13:09:19.241947 4188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:19.241714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:45567)
0806 13:09:19.241860 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:19.241900 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260806 13:09:19.311280 4190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:19.311072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:47881)
0806 13:09:19.311199 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:19.311234 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260806 13:09:19.438585 4191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:19.438401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:37355)
0806 13:09:19.438510 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:19.438544 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260806 13:09:20.242561 4192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:20.242354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:43793)
0806 13:09:20.242479 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:20.242514 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260806 13:09:20.311822 4193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:20.311630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:49687)
0806 13:09:20.311754 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:20.311785 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
I20260806 13:09:20.439144 4194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:20.438927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:42015)
0806 13:09:20.439062 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:20.439100 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260806 13:09:21.067757 4195 raft_consensus.cc:482] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:09:21.067857 4195 raft_consensus.cc:504] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "745b15c1468b4eb7ac30b2a8b80d9c70" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 33975 } } peers { permanent_uuid: "bcb9edc7a2c940f79497ea0296dc048c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 38163 } } peers { permanent_uuid: "39953b23bae048ebbbc20d4e90795dec" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 43559 } }
I20260806 13:09:21.068018 4195 leader_election.cc:288] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975), bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163)
W20260806 13:09:21.068403 3569 leader_election.cc:341] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 745b15c1468b4eb7ac30b2a8b80d9c70 (127.13.224.67:33975): Not found: Tablet not found: 09a382b0316640ebb8b9601348e23448
W20260806 13:09:21.068477 3569 leader_election.cc:341] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bcb9edc7a2c940f79497ea0296dc048c (127.13.224.66:38163): Not found: Tablet not found: 09a382b0316640ebb8b9601348e23448
I20260806 13:09:21.068492 3569 leader_election.cc:302] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39953b23bae048ebbbc20d4e90795dec; no voters: 745b15c1468b4eb7ac30b2a8b80d9c70, bcb9edc7a2c940f79497ea0296dc048c
I20260806 13:09:21.068567 4195 raft_consensus.cc:2726] T 09a382b0316640ebb8b9601348e23448 P 39953b23bae048ebbbc20d4e90795dec [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:09:21.243252 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:21.243002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:49041)
0806 13:09:21.243160 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:21.243201 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260806 13:09:21.312419 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:21.312224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:60001)
0806 13:09:21.312340 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:21.312375 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260806 13:09:21.439721 4198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:21.439512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:57829)
0806 13:09:21.439641 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:21.439674 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260806 13:09:22.243841 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:22.243645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:46413)
0806 13:09:22.243760 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:22.243796 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260806 13:09:22.313053 4200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:22.312842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:50873)
0806 13:09:22.312978 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:22.313014 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260806 13:09:22.440387 4203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:22.440160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:58823)
0806 13:09:22.440306 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:22.440342 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":62,"threads_started":1}
I20260806 13:09:23.244454 4204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:23.244236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:43273)
0806 13:09:23.244371 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:23.244408 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260806 13:09:23.313706 4205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:23.313487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:47879)
0806 13:09:23.313611 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:23.313649 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260806 13:09:23.441089 4207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:23.440847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:51937)
0806 13:09:23.441006 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:23.441043 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":67,"threads_started":1}
I20260806 13:09:24.245116 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:24.244916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.65:34491)
0806 13:09:24.245031 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:24.245067 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260806 13:09:24.314370 4209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:24.314160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.66:38333)
0806 13:09:24.314300 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:24.314333 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260806 13:09:24.438192 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 3503
I20260806 13:09:24.444228 4211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:24.443026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:39465 (local address 127.13.224.67:33377)
0806 13:09:24.444110 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:24.444156 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:39465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1065,"thread_start_us":967,"threads_started":1}
I20260806 13:09:24.451891 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 3751
I20260806 13:09:24.461030 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 3949
2026-08-06T13:09:24Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18289 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:09:24 dist-test-slave-q9h9 krb5kdc[4214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:09:24 dist-test-slave-q9h9 krb5kdc[4214](info): set up 2 sockets
Aug 06 13:09:24 dist-test-slave-q9h9 krb5kdc[4214](info): commencing operation
krb5kdc: starting...
W20260806 13:09:25.580247 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:09:25 dist-test-slave-q9h9 krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:09:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:09:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:09:25.688293 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35327
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:35327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:25.766809 4235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:25.766981 4235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:25.767002 4235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:25.768045 4235 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 13:09:25.768076 4235 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 13:09:25.768087 4235 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 13:09:25.768097 4235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:09:25.768119 4235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:25.768131 4235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:25.768147 4235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:09:25.768160 4235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:09:25.769543 4235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:35327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:25.769726 4235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:25.769886 4235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:25.774748 4244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:25.774761 4243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:25.774765 4247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:25.775357 4235 server_base.cc:1034] running on GCE node
I20260806 13:09:25.775595 4235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:25.775858 4235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:25.776991 4235 hybrid_clock.cc:648] HybridClock initialized: now 1786021765776970 us; error 34 us; skew 500 ppm
Aug 06 13:09:25 dist-test-slave-q9h9 krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021765, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:25.783881 4235 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:09:25.784265 4235 webserver.cc:460] Webserver started at http://127.13.224.126:36351/ using document root <none> and password file <none>
I20260806 13:09:25.784477 4235 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:25.784524 4235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:25.784646 4235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:09:25.785568 4235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "c69a62d1305d4da182f51503bfb8ca67"
format_stamp: "Formatted at 2026-08-06 13:09:25 on dist-test-slave-q9h9"
server_key: "81f6d1d768b084bce8b408c2166e9ae1"
server_key_iv: "53604aad607dff6cee5b3a6218595596"
server_key_version: "encryptionkey@0"
I20260806 13:09:25.785878 4235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "c69a62d1305d4da182f51503bfb8ca67"
format_stamp: "Formatted at 2026-08-06 13:09:25 on dist-test-slave-q9h9"
server_key: "81f6d1d768b084bce8b408c2166e9ae1"
server_key_iv: "53604aad607dff6cee5b3a6218595596"
server_key_version: "encryptionkey@0"
I20260806 13:09:25.787072 4235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:09:25.787784 4258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:25.787950 4235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:09:25.788031 4235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "c69a62d1305d4da182f51503bfb8ca67"
format_stamp: "Formatted at 2026-08-06 13:09:25 on dist-test-slave-q9h9"
server_key: "81f6d1d768b084bce8b408c2166e9ae1"
server_key_iv: "53604aad607dff6cee5b3a6218595596"
server_key_version: "encryptionkey@0"
I20260806 13:09:25.788096 4235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:25.796535 4235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:25.797991 4235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:25.798136 4235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:25.801646 4235 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:35327
I20260806 13:09:25.801692 4336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:35327 every 8 connection(s)
I20260806 13:09:25.801973 4235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:09:25.802193 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 4235
I20260806 13:09:25.802284 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:09:25.802503 14209 external_mini_cluster.cc:1428] Setting key abdcfbfd429aae96c29e22e83c44b0cb
I20260806 13:09:25.802891 4338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:25.805680 4338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Bootstrap starting.
I20260806 13:09:25.806380 4338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:25.806635 4338 log.cc:824] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:25.807294 4338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: No bootstrap required, opened a new log
Aug 06 13:09:25 dist-test-slave-q9h9 krb5kdc[4214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:25.809361 4338 raft_consensus.cc:348] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } }
I20260806 13:09:25.809506 4338 raft_consensus.cc:374] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:25.809556 4338 raft_consensus.cc:729] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c69a62d1305d4da182f51503bfb8ca67, State: Initialized, Role: FOLLOWER
I20260806 13:09:25.809677 4338 consensus_queue.cc:260] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } }
I20260806 13:09:25.809746 4338 raft_consensus.cc:388] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:09:25.809767 4338 raft_consensus.cc:482] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:09:25.809798 4338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:25.810398 4338 raft_consensus.cc:504] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } }
I20260806 13:09:25.810483 4338 leader_election.cc:302] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c69a62d1305d4da182f51503bfb8ca67; no voters:
I20260806 13:09:25.810629 4338 leader_election.cc:288] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:09:25.810686 4345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:09:25.810808 4345 raft_consensus.cc:686] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 1 LEADER]: Becoming Leader. State: Replica: c69a62d1305d4da182f51503bfb8ca67, State: Running, Role: LEADER
I20260806 13:09:25.810879 4338 sys_catalog.cc:564] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:09:25.810886 4345 consensus_queue.cc:237] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } }
I20260806 13:09:25.811192 4345 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c69a62d1305d4da182f51503bfb8ca67. Latest consensus state: current_term: 1 leader_uuid: "c69a62d1305d4da182f51503bfb8ca67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } } }
I20260806 13:09:25.811261 4345 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:25.811400 4348 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c69a62d1305d4da182f51503bfb8ca67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } } }
I20260806 13:09:25.811460 4348 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:25.811573 4352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:09:25.812124 4352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:09:25.812906 4343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:25.802979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58918 (local address 127.13.224.126:35327)
0806 13:09:25.803142 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:25.803151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:25.803178 (+ 27us) server_negotiation.cc:407] Connection header received
0806 13:09:25.803540 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:25.803546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:25.803588 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:25.803690 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:25.804395 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:25.805270 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:25.806295 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:25.806707 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:25.806812 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:25.809209 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:25.809213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:25.809213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:25.809222 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:25.810848 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:25.811234 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:25.811237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:25.811237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:25.811346 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:25.811566 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:25.811567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:25.811567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:25.811866 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:25.812554 (+ 688us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:25.812607 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 13:09:25.812724 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260806 13:09:25.813616 4352 catalog_manager.cc:1347] Generated new cluster ID: 9dc4ff9a21a344d4925be17c81aaa3ff
I20260806 13:09:25.813678 4352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:09:25.826083 4352 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:09:25.826730 4352 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:09:25.829734 4352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Generated new TSK 0
I20260806 13:09:25.829877 4352 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:09:25.853078 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35327
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:25.930866 4371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:25.931048 4371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:25.931080 4371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:25.932181 4371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:25.932220 4371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:25.932263 4371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:09:25.933671 4371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:25.933902 4371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:25.934144 4371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:25.938771 4377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:25.938777 4376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:25.938802 4379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:25.939096 4371 server_base.cc:1034] running on GCE node
I20260806 13:09:25.939275 4371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:25.939481 4371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:25.940609 4371 hybrid_clock.cc:648] HybridClock initialized: now 1786021765940590 us; error 35 us; skew 500 ppm
Aug 06 13:09:25 dist-test-slave-q9h9 krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021765, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:25.942960 4371 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:09:25.943334 4371 webserver.cc:460] Webserver started at http://127.13.224.65:35579/ using document root <none> and password file <none>
I20260806 13:09:25.943537 4371 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:25.943578 4371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:25.943706 4371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:09:25.944624 4371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "a5304c57dea24bb1aeb60411618aa939"
format_stamp: "Formatted at 2026-08-06 13:09:25 on dist-test-slave-q9h9"
server_key: "ac4826dff9c8fb7d4a38640fc256e36e"
server_key_iv: "69ca0e434bd9f4d820559f5910d7117f"
server_key_version: "encryptionkey@0"
I20260806 13:09:25.944944 4371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "a5304c57dea24bb1aeb60411618aa939"
format_stamp: "Formatted at 2026-08-06 13:09:25 on dist-test-slave-q9h9"
server_key: "ac4826dff9c8fb7d4a38640fc256e36e"
server_key_iv: "69ca0e434bd9f4d820559f5910d7117f"
server_key_version: "encryptionkey@0"
I20260806 13:09:25.946130 4371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:09:25.946766 4390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:25.946928 4371 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:25.947010 4371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "a5304c57dea24bb1aeb60411618aa939"
format_stamp: "Formatted at 2026-08-06 13:09:25 on dist-test-slave-q9h9"
server_key: "ac4826dff9c8fb7d4a38640fc256e36e"
server_key_iv: "69ca0e434bd9f4d820559f5910d7117f"
server_key_version: "encryptionkey@0"
I20260806 13:09:25.947081 4371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:25.953977 4371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:25.955595 4371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:25.955711 4371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:25.955899 4371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:25.956158 4371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:09:25.956189 4371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:25.956208 4371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:09:25.956231 4371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:25.961651 4371 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:35807
I20260806 13:09:25.961712 4553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:35807 every 8 connection(s)
I20260806 13:09:25.961951 4371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
Aug 06 13:09:25 dist-test-slave-q9h9 krb5kdc[4214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021765, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:25.967392 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 4371
I20260806 13:09:25.967486 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:09:25.967705 14209 external_mini_cluster.cc:1428] Setting key 86620cf5d3e2d15760124e25e87cc944
I20260806 13:09:25.970166 4343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:25.962571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:53047 (local address 127.13.224.126:35327)
0806 13:09:25.962614 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:25.962616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:25.962795 (+ 179us) server_negotiation.cc:407] Connection header received
0806 13:09:25.963230 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:25.963231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:25.963239 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:25.963286 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:25.964090 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:25.964574 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:25.965448 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:25.965632 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:25.965673 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:25.967639 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:25.967641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:25.967641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:25.967648 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:25.968909 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:25.969346 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:25.969346 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:25.969346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:25.969390 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:25.969613 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:25.969613 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:25.969613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:25.969701 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:25.969742 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:25.970009 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 13:09:25.970104 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:09:25.970415 4558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:25.962536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:53047)
0806 13:09:25.962694 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:25.962732 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:25.963137 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:25.963348 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:25.963351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:25.963856 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:25.964026 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:25.964030 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:25.964631 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:25.964631 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:25.965390 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:25.965393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:25.965683 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:25.965683 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:25.965772 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:25.966229 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:25.966237 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:25.967548 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:25.968982 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:25.968989 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:25.968991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:25.969281 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:25.969438 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:25.969438 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:25.969439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:25.969561 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:25.969749 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:25.969754 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:25.969896 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 13:09:25.970000 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 13:09:25.970195 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260806 13:09:25.970877 4556 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35327
I20260806 13:09:25.970968 4556 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:25.971135 4556 heartbeater.cc:511] Master 127.13.224.126:35327 requested a full tablet report, sending...
I20260806 13:09:25.971642 4286 ts_manager.cc:194] Registered new tserver with Master: a5304c57dea24bb1aeb60411618aa939 (127.13.224.65:35807)
I20260806 13:09:25.972227 4286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:53047
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:09:25.990393 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35327
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:26.068547 4561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:26.068732 4561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:26.068770 4561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:26.069869 4561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:26.069908 4561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:26.069942 4561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:09:26.071338 4561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:26.071549 4561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:26.071755 4561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:26.076366 4573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:26.076411 4561 server_base.cc:1034] running on GCE node
W20260806 13:09:26.076475 4569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:26.076378 4570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:26.076701 4561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:26.076894 4561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:26.078045 4561 hybrid_clock.cc:648] HybridClock initialized: now 1786021766078028 us; error 30 us; skew 500 ppm
Aug 06 13:09:26 dist-test-slave-q9h9 krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021766, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:26.080511 4561 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:09:26.080924 4561 webserver.cc:460] Webserver started at http://127.13.224.66:35309/ using document root <none> and password file <none>
I20260806 13:09:26.081105 4561 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:26.081148 4561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:26.081238 4561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:09:26.082217 4561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "4a6ca56a4470464aafdbe60f10c7ce3c"
format_stamp: "Formatted at 2026-08-06 13:09:26 on dist-test-slave-q9h9"
server_key: "a1ab59117fb9c0251551ede82c3e3b4f"
server_key_iv: "ac14295377b9e038f392e5f9914828d2"
server_key_version: "encryptionkey@0"
I20260806 13:09:26.082515 4561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "4a6ca56a4470464aafdbe60f10c7ce3c"
format_stamp: "Formatted at 2026-08-06 13:09:26 on dist-test-slave-q9h9"
server_key: "a1ab59117fb9c0251551ede82c3e3b4f"
server_key_iv: "ac14295377b9e038f392e5f9914828d2"
server_key_version: "encryptionkey@0"
I20260806 13:09:26.083730 4561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:09:26.084408 4585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:26.084542 4561 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:09:26.084632 4561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "4a6ca56a4470464aafdbe60f10c7ce3c"
format_stamp: "Formatted at 2026-08-06 13:09:26 on dist-test-slave-q9h9"
server_key: "a1ab59117fb9c0251551ede82c3e3b4f"
server_key_iv: "ac14295377b9e038f392e5f9914828d2"
server_key_version: "encryptionkey@0"
I20260806 13:09:26.084695 4561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:26.097934 4561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:26.099563 4561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:26.099700 4561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:26.099905 4561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:26.100198 4561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:09:26.100234 4561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:26.100265 4561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:09:26.100292 4561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:26.106349 4561 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:45183
I20260806 13:09:26.106382 4766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:45183 every 8 connection(s)
I20260806 13:09:26.106664 4561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
Aug 06 13:09:26 dist-test-slave-q9h9 krb5kdc[4214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021766, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:26.114401 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 4561
I20260806 13:09:26.114509 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:09:26.114738 14209 external_mini_cluster.cc:1428] Setting key 8b81733b5593ea0f3f7bc7c206141165
I20260806 13:09:26.114835 4343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.107320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:40667 (local address 127.13.224.126:35327)
0806 13:09:26.107381 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:26.107383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:26.107547 (+ 164us) server_negotiation.cc:407] Connection header received
0806 13:09:26.107988 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:26.107989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:26.107997 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:26.108045 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:26.108862 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.109334 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.110272 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.110451 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.110499 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.112429 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:26.112431 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:26.112431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:26.112438 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:26.113583 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:26.113945 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:26.113946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:26.113947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:26.114003 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:26.114215 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:26.114216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:26.114216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:26.114316 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:26.114364 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:26.114669 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 13:09:26.114735 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 13:09:26.115015 4769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.107273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:40667)
0806 13:09:26.107429 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:26.107467 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:26.107915 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:26.108108 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:26.108111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:26.108623 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:26.108795 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.108800 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.109394 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.109395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.110210 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.110213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.110507 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.110508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.110602 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.110976 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:26.110984 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:26.112339 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:26.113650 (+ 1311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:26.113655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:26.113657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:26.113858 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:26.114086 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:26.114087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:26.114087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:26.114171 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:26.114373 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:26.114379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:26.114532 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 13:09:26.114638 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 13:09:26.114813 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260806 13:09:26.115504 4767 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35327
I20260806 13:09:26.115587 4767 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:26.115746 4767 heartbeater.cc:511] Master 127.13.224.126:35327 requested a full tablet report, sending...
I20260806 13:09:26.116102 4284 ts_manager.cc:194] Registered new tserver with Master: 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183)
I20260806 13:09:26.116523 4284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:40667
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:09:26.136140 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35327
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:26.215243 4773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:26.215425 4773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:26.215446 4773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:26.216578 4773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:26.216616 4773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:26.216641 4773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:09:26.218106 4773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:35327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:26.218339 4773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:26.218540 4773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:26.223138 4782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:26.223146 4788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:26.223189 4786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:26.223356 4773 server_base.cc:1034] running on GCE node
I20260806 13:09:26.223537 4773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:26.223748 4773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:26.224879 4773 hybrid_clock.cc:648] HybridClock initialized: now 1786021766224864 us; error 29 us; skew 500 ppm
Aug 06 13:09:26 dist-test-slave-q9h9 krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021766, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:26.227169 4773 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:09:26.227524 4773 webserver.cc:460] Webserver started at http://127.13.224.67:33819/ using document root <none> and password file <none>
I20260806 13:09:26.227712 4773 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:26.227751 4773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:26.227833 4773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:09:26.228763 4773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "78bb03195a8847dda30e3f8ef6ee1768"
format_stamp: "Formatted at 2026-08-06 13:09:26 on dist-test-slave-q9h9"
server_key: "8e94ab67baca76e358f0f725d930840c"
server_key_iv: "578951dfe00a0fb00259d9afff6130b3"
server_key_version: "encryptionkey@0"
I20260806 13:09:26.229034 4773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "78bb03195a8847dda30e3f8ef6ee1768"
format_stamp: "Formatted at 2026-08-06 13:09:26 on dist-test-slave-q9h9"
server_key: "8e94ab67baca76e358f0f725d930840c"
server_key_iv: "578951dfe00a0fb00259d9afff6130b3"
server_key_version: "encryptionkey@0"
I20260806 13:09:26.230237 4773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:09:26.230854 4799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:26.230993 4773 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:26.231065 4773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "78bb03195a8847dda30e3f8ef6ee1768"
format_stamp: "Formatted at 2026-08-06 13:09:26 on dist-test-slave-q9h9"
server_key: "8e94ab67baca76e358f0f725d930840c"
server_key_iv: "578951dfe00a0fb00259d9afff6130b3"
server_key_version: "encryptionkey@0"
I20260806 13:09:26.231127 4773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:26.264794 4773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:26.266362 4773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:26.266484 4773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:26.266680 4773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:26.266942 4773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:09:26.266974 4773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:26.266993 4773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:09:26.267016 4773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:26.272280 4773 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:43151
I20260806 13:09:26.272346 4970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:43151 every 8 connection(s)
I20260806 13:09:26.272594 4773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
Aug 06 13:09:26 dist-test-slave-q9h9 krb5kdc[4214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021766, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:26.280366 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 4773
I20260806 13:09:26.280475 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:09:26.280591 4343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.273231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:33069 (local address 127.13.224.126:35327)
0806 13:09:26.273277 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:26.273279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:26.273487 (+ 208us) server_negotiation.cc:407] Connection header received
0806 13:09:26.273913 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:26.273914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:26.273923 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:26.273976 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:26.274816 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.275304 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.276244 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.276447 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.276502 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.278302 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:26.278303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:26.278304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:26.278310 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:26.279523 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:26.279837 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:26.279837 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:26.279838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:26.279882 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:26.280043 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:26.280043 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:26.280043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:26.280146 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:26.280190 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:26.280429 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 13:09:26.280506 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:09:26.280681 14209 external_mini_cluster.cc:1428] Setting key a4be814d90e05cc972dadd0ff31aae26
I20260806 13:09:26.280676 4975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.273215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:33069)
0806 13:09:26.273366 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:26.273406 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:26.273836 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:26.274051 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:26.274055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:26.274562 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:26.274733 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.274737 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.275387 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.275387 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.276155 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.276157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.276507 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.276508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.276594 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.276947 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:26.276955 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:26.278228 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:26.279588 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:26.279594 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:26.279596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:26.279785 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:26.279924 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:26.279924 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:26.279924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:26.280004 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:26.280188 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:26.280192 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:26.280286 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 13:09:26.280366 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 13:09:26.280494 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260806 13:09:26.281160 4971 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35327
I20260806 13:09:26.281247 4971 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:26.281402 4971 heartbeater.cc:511] Master 127.13.224.126:35327 requested a full tablet report, sending...
I20260806 13:09:26.281721 4284 ts_manager.cc:194] Registered new tserver with Master: 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151)
I20260806 13:09:26.282114 4284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:33069
I20260806 13:09:26.292177 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:09:26.298300 4343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.293442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58924 (local address 127.13.224.126:35327)
0806 13:09:26.293489 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:26.293491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:26.293565 (+ 74us) server_negotiation.cc:407] Connection header received
0806 13:09:26.293609 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:26.293609 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:26.293616 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:26.293664 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:26.294192 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.294624 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.295308 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.295472 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.295513 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.296491 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:26.296492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:26.296492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:26.296496 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:26.297457 (+ 961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:26.297720 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:26.297721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:26.297721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:26.297770 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:26.297888 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:26.297888 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:26.297889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:26.297998 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:26.298107 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:26.298192 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 13:09:26.298245 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 13:09:26.299711 4284 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:09:26.301136 4284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:09:26.306934 4989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.303347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:35807 (local address 127.0.0.1:44300)
0806 13:09:26.303459 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:26.303467 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:26.303501 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:26.303962 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:26.303963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:26.303964 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:26.304007 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.304009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.304857 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.304858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.305726 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.305729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.306645 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.306654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.306729 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.306743 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 13:09:26.306810 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 13:09:26.306865 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":3,"thread_start_us":75,"threads_started":1}
I20260806 13:09:26.307029 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.303077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.13.224.67:43151)
0806 13:09:26.303236 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:26.303240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:26.303265 (+ 25us) server_negotiation.cc:407] Connection header received
0806 13:09:26.303300 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:26.303302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:26.303317 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:26.303388 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:26.303577 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.304621 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.306008 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.306680 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.306774 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.306827 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:26.306918 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 13:09:26.306954 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260806 13:09:26.307041 4984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.302993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:43151 (local address 127.0.0.1:60348)
0806 13:09:26.303106 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:26.303142 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:26.303201 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:26.303459 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:26.303462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:26.303464 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:26.303522 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.303524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.304725 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.304726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.305937 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.305940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.306784 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.306786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.306848 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.306852 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:09:26.306906 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 13:09:26.306944 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":38,"thread_start_us":72,"threads_started":1}
I20260806 13:09:26.307361 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.303552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44300 (local address 127.13.224.65:35807)
0806 13:09:26.303738 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:26.303742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:26.303753 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:09:26.303791 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:26.303793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:26.303809 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:26.303879 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:26.304065 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.304754 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.305814 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.306562 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.306636 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.306692 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:26.307237 (+ 545us) server_negotiation.cc:299] Negotiation successful
0806 13:09:26.307284 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":83,"threads_started":1}
I20260806 13:09:26.307641 4865 tablet_service.cc:1467] Processing CreateTablet for tablet 5e11ee6dfaf046a3b5080c894a058c4b (DEFAULT_TABLE table=test-table [id=2db6e75923b546b091e1af38fc96ef0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:26.307982 4865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e11ee6dfaf046a3b5080c894a058c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:26.308028 4465 tablet_service.cc:1467] Processing CreateTablet for tablet 5e11ee6dfaf046a3b5080c894a058c4b (DEFAULT_TABLE table=test-table [id=2db6e75923b546b091e1af38fc96ef0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:26.308360 4465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e11ee6dfaf046a3b5080c894a058c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:26.310146 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.303219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:45183 (local address 127.0.0.1:49232)
0806 13:09:26.303313 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:26.303323 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:26.303369 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:26.303872 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:26.303873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:26.303875 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:26.303919 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.303920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.305134 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.305135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.306197 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.306199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.306889 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.306890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.306946 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.306949 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:09:26.306978 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 13:09:26.310095 (+ 3117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":18,"thread_start_us":52,"threads_started":1}
I20260806 13:09:26.310503 4990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.303531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49232 (local address 127.13.224.66:45183)
0806 13:09:26.303692 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:26.303696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:26.303705 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:09:26.303742 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:26.303756 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:26.303771 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:26.303981 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:26.304050 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.305040 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.306267 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.306805 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.310309 (+ 3504us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.310388 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:26.310409 (+ 21us) server_negotiation.cc:299] Negotiation successful
0806 13:09:26.310442 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260806 13:09:26.311084 4665 tablet_service.cc:1467] Processing CreateTablet for tablet 5e11ee6dfaf046a3b5080c894a058c4b (DEFAULT_TABLE table=test-table [id=2db6e75923b546b091e1af38fc96ef0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:26.311412 4665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e11ee6dfaf046a3b5080c894a058c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:26.311578 4995 tablet_bootstrap.cc:492] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Bootstrap starting.
I20260806 13:09:26.312124 4995 tablet_bootstrap.cc:654] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:26.312609 4995 log.cc:824] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:26.313208 4995 tablet_bootstrap.cc:492] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: No bootstrap required, opened a new log
I20260806 13:09:26.313251 4995 ts_tablet_manager.cc:1397] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:09:26.313901 4999 tablet_bootstrap.cc:492] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c: Bootstrap starting.
I20260806 13:09:26.314280 4992 tablet_bootstrap.cc:492] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768: Bootstrap starting.
I20260806 13:09:26.314533 4999 tablet_bootstrap.cc:654] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:26.314832 4992 tablet_bootstrap.cc:654] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:26.314987 4995 raft_consensus.cc:348] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:26.315155 4999 log.cc:824] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:26.315178 4995 raft_consensus.cc:374] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:26.315222 4995 raft_consensus.cc:729] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5304c57dea24bb1aeb60411618aa939, State: Initialized, Role: FOLLOWER
I20260806 13:09:26.315330 4992 log.cc:824] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:26.315368 4995 consensus_queue.cc:260] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:26.315629 4995 ts_tablet_manager.cc:1428] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 13:09:26.315763 4556 heartbeater.cc:503] Master 127.13.224.126:35327 was elected leader, sending a full tablet report...
I20260806 13:09:26.315973 4999 tablet_bootstrap.cc:492] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c: No bootstrap required, opened a new log
I20260806 13:09:26.316037 4999 ts_tablet_manager.cc:1397] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:09:26.316576 4992 tablet_bootstrap.cc:492] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768: No bootstrap required, opened a new log
I20260806 13:09:26.316634 4992 ts_tablet_manager.cc:1397] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:09:26.317662 4999 raft_consensus.cc:348] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:26.317795 4999 raft_consensus.cc:374] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:26.317814 4999 raft_consensus.cc:729] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a6ca56a4470464aafdbe60f10c7ce3c, State: Initialized, Role: FOLLOWER
I20260806 13:09:26.317924 4999 consensus_queue.cc:260] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:26.318173 4999 ts_tablet_manager.cc:1428] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 13:09:26.318219 4767 heartbeater.cc:503] Master 127.13.224.126:35327 was elected leader, sending a full tablet report...
I20260806 13:09:26.318498 4992 raft_consensus.cc:348] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:26.318620 4992 raft_consensus.cc:374] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:26.318651 4992 raft_consensus.cc:729] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bb03195a8847dda30e3f8ef6ee1768, State: Initialized, Role: FOLLOWER
I20260806 13:09:26.318775 4992 consensus_queue.cc:260] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:26.319041 4971 heartbeater.cc:503] Master 127.13.224.126:35327 was elected leader, sending a full tablet report...
I20260806 13:09:26.319279 4992 ts_tablet_manager.cc:1428] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260806 13:09:26.357174 4768 tablet.cc:2369] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:09:26.367223 5003 raft_consensus.cc:482] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:09:26.367334 5003 raft_consensus.cc:504] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:26.367674 5003 leader_election.cc:288] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151), a5304c57dea24bb1aeb60411618aa939 (127.13.224.65:35807)
I20260806 13:09:26.371206 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.367903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:40877 (local address 127.13.224.67:43151)
0806 13:09:26.367968 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:26.367971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:26.367980 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:09:26.368008 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:26.368008 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:26.368017 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:26.368068 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:26.368226 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.368966 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.370309 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.370877 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.370950 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.370958 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:26.371101 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 13:09:26.371137 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 13:09:26.371207 4991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.367974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:40697 (local address 127.13.224.65:35807)
0806 13:09:26.368016 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:26.368018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:26.368132 (+ 114us) server_negotiation.cc:407] Connection header received
0806 13:09:26.368184 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:26.368185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:26.368196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:26.368248 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:26.368354 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.369046 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.370303 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.370879 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.370951 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.370958 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:26.371101 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 13:09:26.371137 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:09:26.371235 4769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.367813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:43151 (local address 127.13.224.66:40877)
0806 13:09:26.367863 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:26.367874 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:26.367925 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:26.368090 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:26.368091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:26.368093 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:26.368164 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.368166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.369032 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.369032 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.370232 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.370235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.370952 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.370952 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.371033 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.371039 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 13:09:26.371090 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 13:09:26.371133 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":8}
I20260806 13:09:26.371243 5005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.367918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:35807 (local address 127.13.224.66:40697)
0806 13:09:26.368059 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:26.368068 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:26.368120 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:26.368258 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:26.368258 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:26.368259 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:26.368304 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.368305 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.369112 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.369112 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.370232 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:26.370235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.370952 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.370953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:26.371033 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.371039 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 13:09:26.371090 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 13:09:26.371133 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":11,"thread_start_us":86,"threads_started":1}
I20260806 13:09:26.371613 4493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e11ee6dfaf046a3b5080c894a058c4b" candidate_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5304c57dea24bb1aeb60411618aa939" is_pre_election: true
I20260806 13:09:26.371661 4897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e11ee6dfaf046a3b5080c894a058c4b" candidate_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bb03195a8847dda30e3f8ef6ee1768" is_pre_election: true
I20260806 13:09:26.371739 4493 raft_consensus.cc:2454] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a6ca56a4470464aafdbe60f10c7ce3c in term 0.
I20260806 13:09:26.371795 4897 raft_consensus.cc:2454] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a6ca56a4470464aafdbe60f10c7ce3c in term 0.
I20260806 13:09:26.371981 4588 leader_election.cc:302] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a6ca56a4470464aafdbe60f10c7ce3c, a5304c57dea24bb1aeb60411618aa939; no voters:
I20260806 13:09:26.372095 5003 raft_consensus.cc:2781] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:09:26.372152 5003 raft_consensus.cc:482] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:09:26.372176 5003 raft_consensus.cc:3037] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:26.372833 5003 raft_consensus.cc:504] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:26.372934 5003 leader_election.cc:288] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [CANDIDATE]: Term 1 election: Requested vote from peers 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151), a5304c57dea24bb1aeb60411618aa939 (127.13.224.65:35807)
I20260806 13:09:26.373098 4897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e11ee6dfaf046a3b5080c894a058c4b" candidate_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bb03195a8847dda30e3f8ef6ee1768"
I20260806 13:09:26.373142 4493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e11ee6dfaf046a3b5080c894a058c4b" candidate_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5304c57dea24bb1aeb60411618aa939"
I20260806 13:09:26.373198 4493 raft_consensus.cc:3037] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:26.373190 4897 raft_consensus.cc:3037] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:26.373880 4897 raft_consensus.cc:2454] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a6ca56a4470464aafdbe60f10c7ce3c in term 1.
I20260806 13:09:26.373885 4493 raft_consensus.cc:2454] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a6ca56a4470464aafdbe60f10c7ce3c in term 1.
I20260806 13:09:26.374054 4588 leader_election.cc:302] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a6ca56a4470464aafdbe60f10c7ce3c, 78bb03195a8847dda30e3f8ef6ee1768; no voters:
I20260806 13:09:26.374145 5003 raft_consensus.cc:2781] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:09:26.374313 5003 raft_consensus.cc:686] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [term 1 LEADER]: Becoming Leader. State: Replica: 4a6ca56a4470464aafdbe60f10c7ce3c, State: Running, Role: LEADER
I20260806 13:09:26.374394 5003 consensus_queue.cc:237] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:26.375162 4284 catalog_manager.cc:5495] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66). New cstate: current_term: 1 leader_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } health_report { overall_health: UNKNOWN } } }
I20260806 13:09:26.381062 4990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.379225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49240 (local address 127.13.224.66:45183)
0806 13:09:26.379271 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:26.379272 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:26.379279 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:09:26.379304 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:26.379305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:26.379319 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:26.379373 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:09:26.379461 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.379880 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.380540 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:26.380706 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:26.380752 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:26.380844 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:09:26.380926 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:09:26.380975 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:26.380987 (+ 12us) server_negotiation.cc:299] Negotiation successful
0806 13:09:26.381001 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 13:09:26.382527 4493 raft_consensus.cc:1261] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Refusing update from remote peer 4a6ca56a4470464aafdbe60f10c7ce3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:09:26.382541 4897 raft_consensus.cc:1261] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768 [term 1 FOLLOWER]: Refusing update from remote peer 4a6ca56a4470464aafdbe60f10c7ce3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:09:26.382819 5003 consensus_queue.cc:1035] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:09:26.382951 5007 consensus_queue.cc:1035] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:09:26.385365 5015 mvcc.cc:205] Tried to move back new op lower bound from 7315545155100815360 to 7315545155070038016. Current Snapshot: MvccSnapshot[applied={T|T < 7315545155100815360}]
I20260806 13:09:26.387049 4286 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:09:26.387125 4286 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:09:26.387773 4286 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Sending DeleteTablet for 3 replicas of tablet 5e11ee6dfaf046a3b5080c894a058c4b
I20260806 13:09:26.388046 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 4371
I20260806 13:09:26.388123 4865 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e11ee6dfaf046a3b5080c894a058c4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:26 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:60348
I20260806 13:09:26.388151 4465 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e11ee6dfaf046a3b5080c894a058c4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:26 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:44300
I20260806 13:09:26.388201 4665 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e11ee6dfaf046a3b5080c894a058c4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:26 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:49232
I20260806 13:09:26.403865 5024 tablet_replica.cc:330] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768: stopping tablet replica
I20260806 13:09:26.403996 5024 raft_consensus.cc:2229] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:09:26.404120 5024 raft_consensus.cc:2258] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:09:26.404120 5025 tablet_replica.cc:330] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c: stopping tablet replica
I20260806 13:09:26.404279 5025 raft_consensus.cc:2229] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:09:26.404428 5025 raft_consensus.cc:2258] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:09:26.404779 5024 ts_tablet_manager.cc:1905] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:09:26.405040 5025 ts_tablet_manager.cc:1905] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:09:26.406466 5024 ts_tablet_manager.cc:1918] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:09:26.406548 5024 log.cc:1196] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/5e11ee6dfaf046a3b5080c894a058c4b
I20260806 13:09:26.406781 5025 ts_tablet_manager.cc:1918] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:09:26.406842 5024 ts_tablet_manager.cc:1939] T 5e11ee6dfaf046a3b5080c894a058c4b P 78bb03195a8847dda30e3f8ef6ee1768: Deleting consensus metadata
I20260806 13:09:26.406941 5025 log.cc:1196] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/5e11ee6dfaf046a3b5080c894a058c4b
I20260806 13:09:26.407274 5025 ts_tablet_manager.cc:1939] T 5e11ee6dfaf046a3b5080c894a058c4b P 4a6ca56a4470464aafdbe60f10c7ce3c: Deleting consensus metadata
I20260806 13:09:26.407269 4261 catalog_manager.cc:4841] TS 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151): tablet 5e11ee6dfaf046a3b5080c894a058c4b (table test-table [id=2db6e75923b546b091e1af38fc96ef0e]) successfully deleted
I20260806 13:09:26.407681 4261 catalog_manager.cc:4841] TS 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183): tablet 5e11ee6dfaf046a3b5080c894a058c4b (table test-table [id=2db6e75923b546b091e1af38fc96ef0e]) successfully deleted
W20260806 13:09:26.409703 4261 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:35807 (error 108)
I20260806 13:09:26.409888 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 4561
I20260806 13:09:26.411188 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:26.410033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:35807 (local address 127.0.0.1:44312)
0806 13:09:26.411108 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:26.411156 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:35807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1050}
W20260806 13:09:26.415067 4261 catalog_manager.cc:4584] TS a5304c57dea24bb1aeb60411618aa939 (127.13.224.65:35807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e11ee6dfaf046a3b5080c894a058c4b: Network error: Client connection negotiation failed: client connection to 127.13.224.65:35807: connect: Connection refused (error 111)
I20260806 13:09:26.427230 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 4773
I20260806 13:09:26.436017 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 4235
I20260806 13:09:46.447846 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35327
--webserver_interface=127.13.224.126
--webserver_port=36351
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:35327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:46.530275 5029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:46.530463 5029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:46.530494 5029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:46.531595 5029 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 13:09:46.531633 5029 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 13:09:46.531651 5029 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 13:09:46.531665 5029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:09:46.531682 5029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:46.531704 5029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:46.531728 5029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:09:46.531764 5029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:09:46.533202 5029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:35327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=36351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:46.533442 5029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:46.533643 5029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:46.542236 5040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:46.542311 5029 server_base.cc:1034] running on GCE node
W20260806 13:09:46.542233 5039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:46.542253 5042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:46.542706 5029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:46.543018 5029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:46.544168 5029 hybrid_clock.cc:648] HybridClock initialized: now 1786021786544150 us; error 29 us; skew 500 ppm
Aug 06 13:09:46 dist-test-slave-q9h9 krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021786, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:46.546666 5029 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:09:46.547088 5029 webserver.cc:460] Webserver started at http://127.13.224.126:36351/ using document root <none> and password file <none>
I20260806 13:09:46.547297 5029 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:46.547360 5029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:46.548677 5029 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:09:46.549315 5053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:46.549467 5029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:09:46.549568 5029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "c69a62d1305d4da182f51503bfb8ca67"
format_stamp: "Formatted at 2026-08-06 13:09:25 on dist-test-slave-q9h9"
server_key: "81f6d1d768b084bce8b408c2166e9ae1"
server_key_iv: "53604aad607dff6cee5b3a6218595596"
server_key_version: "encryptionkey@0"
I20260806 13:09:46.549918 5029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:46.559609 5029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:46.561654 5029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:46.561791 5029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:46.565847 5029 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:35327
I20260806 13:09:46.565877 5129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:35327 every 8 connection(s)
I20260806 13:09:46.566253 5029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:09:46.567613 5131 sys_catalog.cc:263] Verifying existing consensus state
I20260806 13:09:46.568260 5131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Bootstrap starting.
I20260806 13:09:46.573907 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 5029
I20260806 13:09:46.574215 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:35807
--local_ip_for_outbound_sockets=127.13.224.65
--tserver_master_addrs=127.13.224.126:35327
--webserver_port=35579
--webserver_interface=127.13.224.65
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:09:46.576715 5131 log.cc:824] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:46.578521 5131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:09:46.578864 5131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Bootstrap complete.
I20260806 13:09:46.580860 5131 raft_consensus.cc:348] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } }
I20260806 13:09:46.581069 5131 raft_consensus.cc:729] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c69a62d1305d4da182f51503bfb8ca67, State: Initialized, Role: FOLLOWER
I20260806 13:09:46.581200 5131 consensus_queue.cc:260] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } }
I20260806 13:09:46.581262 5131 raft_consensus.cc:388] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:09:46.581306 5131 raft_consensus.cc:482] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:09:46.581354 5131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 1 FOLLOWER]: Advancing to term 2
I20260806 13:09:46.582360 5131 raft_consensus.cc:504] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } }
I20260806 13:09:46.582445 5131 leader_election.cc:302] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c69a62d1305d4da182f51503bfb8ca67; no voters:
I20260806 13:09:46.582598 5131 leader_election.cc:288] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 13:09:46.582644 5135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 13:09:46.582803 5135 raft_consensus.cc:686] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 2 LEADER]: Becoming Leader. State: Replica: c69a62d1305d4da182f51503bfb8ca67, State: Running, Role: LEADER
I20260806 13:09:46.582861 5131 sys_catalog.cc:564] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:09:46.582919 5135 consensus_queue.cc:237] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } }
I20260806 13:09:46.583173 5135 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c69a62d1305d4da182f51503bfb8ca67. Latest consensus state: current_term: 2 leader_uuid: "c69a62d1305d4da182f51503bfb8ca67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } } }
I20260806 13:09:46.583377 5136 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c69a62d1305d4da182f51503bfb8ca67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } } }
I20260806 13:09:46.583454 5135 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:46.583482 5136 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: This master's current role is: LEADER
I20260806 13:09:46.583933 5147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:09:46.584784 5147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:09:46.584905 5147 catalog_manager.cc:1259] Loaded cluster ID: 9dc4ff9a21a344d4925be17c81aaa3ff
I20260806 13:09:46.584949 5147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:09:46.586205 5147 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:09:46.586406 5147 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Loaded TSK: 0
I20260806 13:09:46.599082 5147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Generated new TSK 1
I20260806 13:09:46.599171 5147 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 13:09:46.661697 5133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:46.661896 5133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:46.661919 5133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:46.663057 5133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:46.663103 5133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:46.663129 5133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:09:46.664587 5133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:35807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=35579
--tserver_master_addrs=127.13.224.126:35327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:46.664804 5133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:46.665031 5133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:46.670899 5174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:46.670912 5173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:46.671082 5133 server_base.cc:1034] running on GCE node
W20260806 13:09:46.671209 5177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:46.671399 5133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:46.671641 5133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:46.672811 5133 hybrid_clock.cc:648] HybridClock initialized: now 1786021786672796 us; error 33 us; skew 500 ppm
Aug 06 13:09:46 dist-test-slave-q9h9 krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021786, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:46.675278 5133 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:09:46.675735 5133 webserver.cc:460] Webserver started at http://127.13.224.65:35579/ using document root <none> and password file <none>
I20260806 13:09:46.675938 5133 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:46.675998 5133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:46.677313 5133 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:09:46.677961 5188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:46.678138 5133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:09:46.678236 5133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "a5304c57dea24bb1aeb60411618aa939"
format_stamp: "Formatted at 2026-08-06 13:09:25 on dist-test-slave-q9h9"
server_key: "ac4826dff9c8fb7d4a38640fc256e36e"
server_key_iv: "69ca0e434bd9f4d820559f5910d7117f"
server_key_version: "encryptionkey@0"
I20260806 13:09:46.678653 5133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:46.714138 5133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:46.716116 5133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:46.716236 5133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:46.716493 5133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:46.717005 5203 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 13:09:46.718482 5133 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 13:09:46.718526 5133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 13:09:46.718559 5133 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 13:09:46.719190 5133 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 13:09:46.719223 5133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 13:09:46.719334 5203 tablet_bootstrap.cc:492] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Bootstrap starting.
I20260806 13:09:46.725636 5133 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:35807
I20260806 13:09:46.725737 5364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:35807 every 8 connection(s)
I20260806 13:09:46.726044 5133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:09:46.729411 5203 log.cc:824] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:46.729974 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 5133
I20260806 13:09:46.730307 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:45183
--local_ip_for_outbound_sockets=127.13.224.66
--tserver_master_addrs=127.13.224.126:35327
--webserver_port=35309
--webserver_interface=127.13.224.66
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:09:46.731820 5203 tablet_bootstrap.cc:492] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:09:46.732177 5203 tablet_bootstrap.cc:492] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Bootstrap complete.
Aug 06 13:09:46 dist-test-slave-q9h9 krb5kdc[4214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021786, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:46.733368 5203 ts_tablet_manager.cc:1397] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.008s
I20260806 13:09:46.734530 5203 raft_consensus.cc:348] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:46.734835 5203 raft_consensus.cc:729] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5304c57dea24bb1aeb60411618aa939, State: Initialized, Role: FOLLOWER
I20260806 13:09:46.734987 5203 consensus_queue.cc:260] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:46.735271 5203 ts_tablet_manager.cc:1428] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:09:46.736819 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.726814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:37867 (local address 127.13.224.126:35327)
0806 13:09:46.727008 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:46.727017 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:46.727056 (+ 39us) server_negotiation.cc:407] Connection header received
0806 13:09:46.727579 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:46.727584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:46.727633 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:46.727747 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:46.728334 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.729238 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.730597 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.731197 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.731325 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:46.733498 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:46.733502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:46.733502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:46.733513 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:46.735201 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:46.735647 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:46.735650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:46.735650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:46.735740 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:46.735961 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:46.735962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:46.735963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:46.736174 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:46.736255 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:46.736522 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 13:09:46.736641 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
I20260806 13:09:46.736819 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.726698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:37867)
0806 13:09:46.726865 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:46.726906 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:46.727380 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:46.727796 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:46.727800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:46.728048 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:46.728224 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:46.728229 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.729366 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.729367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:46.730486 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:46.730489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.731333 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.731335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:46.731483 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:46.732017 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:46.732025 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:46.733391 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:46.735290 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:46.735294 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:46.735296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:46.735567 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:46.735800 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:46.735801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:46.735802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:46.735906 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:46.736237 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:46.736243 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:46.736373 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 13:09:46.736457 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 13:09:46.736617 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260806 13:09:46.737538 5365 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35327
I20260806 13:09:46.737639 5365 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:46.737813 5365 heartbeater.cc:511] Master 127.13.224.126:35327 requested a full tablet report, sending...
I20260806 13:09:46.738583 5075 ts_manager.cc:194] Registered new tserver with Master: a5304c57dea24bb1aeb60411618aa939 (127.13.224.65:35807)
I20260806 13:09:46.739751 5075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:37867
I20260806 13:09:46.740563 5365 heartbeater.cc:503] Master 127.13.224.126:35327 was elected leader, sending a full tablet report...
I20260806 13:09:46.743803 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.739183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:35807 (local address 127.0.0.1:41934)
0806 13:09:46.740061 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:46.740095 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:46.740168 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:46.740570 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:46.740573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:46.740576 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:46.740634 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:46.740635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.741958 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.741959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:46.742762 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:46.742765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.743604 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.743605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:46.743666 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:46.743676 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 13:09:46.743718 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 13:09:46.743756 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"thread_start_us":94,"threads_started":1}
I20260806 13:09:46.743829 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.739241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41934 (local address 127.13.224.65:35807)
0806 13:09:46.740306 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:46.740310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:46.740319 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:09:46.740351 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:46.740353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:46.740460 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:46.740543 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:46.740712 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.741882 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.742839 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.743531 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.743598 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:46.743647 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:46.743729 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 13:09:46.743762 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1033,"thread_start_us":71,"threads_started":1}
I20260806 13:09:46.744154 5264 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e11ee6dfaf046a3b5080c894a058c4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:26 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:41934
I20260806 13:09:46.744582 5377 tablet_replica.cc:330] stopping tablet replica
I20260806 13:09:46.744673 5377 raft_consensus.cc:2229] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:09:46.744735 5377 raft_consensus.cc:2258] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:09:46.744992 5377 ts_tablet_manager.cc:1905] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:09:46.746809 5377 ts_tablet_manager.cc:1918] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:09:46.746883 5377 log.cc:1196] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal/wals/5e11ee6dfaf046a3b5080c894a058c4b
I20260806 13:09:46.747067 5377 ts_tablet_manager.cc:1939] T 5e11ee6dfaf046a3b5080c894a058c4b P a5304c57dea24bb1aeb60411618aa939: Deleting consensus metadata
I20260806 13:09:46.747503 5056 catalog_manager.cc:4841] TS a5304c57dea24bb1aeb60411618aa939 (127.13.224.65:35807): tablet 5e11ee6dfaf046a3b5080c894a058c4b (table test-table [id=2db6e75923b546b091e1af38fc96ef0e]) successfully deleted
W20260806 13:09:46.814901 5371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:46.815092 5371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:46.815122 5371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:46.816246 5371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:46.816286 5371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:46.816320 5371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:09:46.817746 5371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:45183
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=35309
--tserver_master_addrs=127.13.224.126:35327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:46.817966 5371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:46.818192 5371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:46.822773 5387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:46.822820 5386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:46.822858 5371 server_base.cc:1034] running on GCE node
W20260806 13:09:46.822830 5391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:46.823156 5371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:46.823362 5371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:46.824486 5371 hybrid_clock.cc:648] HybridClock initialized: now 1786021786824476 us; error 27 us; skew 500 ppm
Aug 06 13:09:46 dist-test-slave-q9h9 krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021786, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:46.826781 5371 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:09:46.827163 5371 webserver.cc:460] Webserver started at http://127.13.224.66:35309/ using document root <none> and password file <none>
I20260806 13:09:46.827356 5371 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:46.827415 5371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:46.828626 5371 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:09:46.829115 5403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:46.829268 5371 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:46.829346 5371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "4a6ca56a4470464aafdbe60f10c7ce3c"
format_stamp: "Formatted at 2026-08-06 13:09:26 on dist-test-slave-q9h9"
server_key: "a1ab59117fb9c0251551ede82c3e3b4f"
server_key_iv: "ac14295377b9e038f392e5f9914828d2"
server_key_version: "encryptionkey@0"
I20260806 13:09:46.829625 5371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:46.838418 5371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:46.840361 5371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:46.840488 5371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:46.840688 5371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:46.840950 5371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:09:46.840982 5371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:46.841008 5371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:09:46.841032 5371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:46.846568 5371 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:45183
I20260806 13:09:46.846625 5569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:45183 every 8 connection(s)
I20260806 13:09:46.846954 5371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
Aug 06 13:09:46 dist-test-slave-q9h9 krb5kdc[4214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021786, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:46.854784 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.847584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:45191 (local address 127.13.224.126:35327)
0806 13:09:46.847641 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:46.847643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:46.847810 (+ 167us) server_negotiation.cc:407] Connection header received
0806 13:09:46.848224 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:46.848225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:46.848237 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:46.848289 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:46.849079 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.849566 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.850515 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.850734 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.850792 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:46.852607 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:46.852609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:46.852609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:46.852616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:46.853714 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:46.854100 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:46.854100 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:46.854101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:46.854149 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:46.854328 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:46.854328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:46.854328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:46.854417 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:46.854459 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:46.854657 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 13:09:46.854722 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 13:09:46.854928 5572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.847562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:45191)
0806 13:09:46.847702 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:46.847739 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:46.848154 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:46.848337 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:46.848340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:46.848845 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:46.849013 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:46.849017 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.849625 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.849626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:46.850425 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:46.850427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.850803 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.850804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:46.850894 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:46.851290 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:46.851297 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:46.852525 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:46.853777 (+ 1252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:46.853783 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:46.853785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:46.853976 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:46.854190 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:46.854190 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:46.854190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:46.854287 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:46.854460 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:46.854463 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:46.854568 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 13:09:46.854642 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 13:09:46.854748 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260806 13:09:46.855358 5570 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35327
I20260806 13:09:46.855436 5570 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:46.855564 5570 heartbeater.cc:511] Master 127.13.224.126:35327 requested a full tablet report, sending...
I20260806 13:09:46.855862 5075 ts_manager.cc:194] Registered new tserver with Master: 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183)
I20260806 13:09:46.856199 5075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:45191
I20260806 13:09:46.856957 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 5371
I20260806 13:09:46.857219 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:43151
--local_ip_for_outbound_sockets=127.13.224.67
--tserver_master_addrs=127.13.224.126:35327
--webserver_port=33819
--webserver_interface=127.13.224.67
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:09:46.937732 5573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:09:46.937923 5573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:09:46.937943 5573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:09:46.939040 5573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:09:46.939071 5573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:09:46.939095 5573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:09:46.940466 5573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:43151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=33819
--tserver_master_addrs=127.13.224.126:35327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:09:46.940631 5573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:09:46.940810 5573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:09:46.945413 5583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:46.945421 5580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:09:46.945407 5585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:09:46.945631 5573 server_base.cc:1034] running on GCE node
I20260806 13:09:46.945811 5573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:09:46.946002 5573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:09:46.947154 5573 hybrid_clock.cc:648] HybridClock initialized: now 1786021786947136 us; error 35 us; skew 500 ppm
Aug 06 13:09:46 dist-test-slave-q9h9 krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021786, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:09:46.949668 5573 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:09:46.950073 5573 webserver.cc:460] Webserver started at http://127.13.224.67:33819/ using document root <none> and password file <none>
I20260806 13:09:46.950268 5573 fs_manager.cc:359] Metadata directory not provided
I20260806 13:09:46.950321 5573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:09:46.951480 5573 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:09:46.952009 5597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:46.952158 5573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:46.952241 5573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "78bb03195a8847dda30e3f8ef6ee1768"
format_stamp: "Formatted at 2026-08-06 13:09:26 on dist-test-slave-q9h9"
server_key: "8e94ab67baca76e358f0f725d930840c"
server_key_iv: "578951dfe00a0fb00259d9afff6130b3"
server_key_version: "encryptionkey@0"
I20260806 13:09:46.952519 5573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:09:46.970149 5573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:09:46.972045 5573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:09:46.972150 5573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:09:46.972348 5573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:09:46.972590 5573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:09:46.972620 5573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:46.972640 5573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:09:46.972656 5573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:46.978223 5573 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:43151
I20260806 13:09:46.978287 5756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:43151 every 8 connection(s)
I20260806 13:09:46.978580 5573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:09:46.982471 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 5573
Aug 06 13:09:46 dist-test-slave-q9h9 krb5kdc[4214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021786, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:09:46.987718 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.979279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:50641 (local address 127.13.224.126:35327)
0806 13:09:46.979343 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:46.979346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:46.979504 (+ 158us) server_negotiation.cc:407] Connection header received
0806 13:09:46.979965 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:46.979966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:46.979976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:46.980024 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:46.980866 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.981345 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.982321 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.982562 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.982618 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:46.984964 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:46.984966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:46.984966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:46.984973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:46.986388 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:46.986828 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:46.986828 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:46.986829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:46.986875 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:46.987101 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:46.987102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:46.987102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:46.987228 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:46.987289 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:46.987547 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 13:09:46.987628 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 13:09:46.987862 5760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.979213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:50641)
0806 13:09:46.979389 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:46.979422 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:46.979888 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:46.980076 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:46.980079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:46.980598 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:09:46.980786 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:46.980791 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.981405 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.981406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:46.982248 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:46.982251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.982650 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.982651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:46.982786 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:46.983333 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:09:46.983341 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:09:46.984882 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:09:46.986463 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:46.986471 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:46.986473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:46.986762 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:46.986932 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:09:46.986933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:09:46.986933 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:09:46.987043 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:09:46.987293 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:09:46.987301 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:09:46.987427 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 13:09:46.987514 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 13:09:46.987659 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260806 13:09:46.988481 5758 heartbeater.cc:347] Connected to a master server at 127.13.224.126:35327
I20260806 13:09:46.988564 5758 heartbeater.cc:464] Registering TS with master...
I20260806 13:09:46.988731 5758 heartbeater.cc:511] Master 127.13.224.126:35327 requested a full tablet report, sending...
I20260806 13:09:46.989133 5075 ts_manager.cc:194] Registered new tserver with Master: 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151)
I20260806 13:09:46.989542 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.982830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37812 (local address 127.13.224.126:35327)
0806 13:09:46.982940 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:46.982942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:46.983126 (+ 184us) server_negotiation.cc:407] Connection header received
0806 13:09:46.983191 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:46.983191 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:46.983203 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:46.983252 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:46.983863 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.984577 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.985496 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.985704 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.985761 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:46.986949 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:46.986951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:46.986951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:46.986957 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:46.988501 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:46.988799 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:46.988800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:46.988801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:46.988856 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:46.989056 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:46.989056 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:46.989056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:46.989180 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:46.989348 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:46.989384 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 13:09:46.989455 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
I20260806 13:09:46.989559 5075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:50641
I20260806 13:09:46.990202 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:09:46.992419 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.990524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37818 (local address 127.13.224.126:35327)
0806 13:09:46.990556 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:46.990557 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:46.990642 (+ 85us) server_negotiation.cc:407] Connection header received
0806 13:09:46.990696 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:46.990696 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:46.990723 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:46.990781 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:09:46.990852 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.991268 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.991917 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.992086 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.992125 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:46.992217 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:09:46.992299 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:09:46.992347 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:46.992363 (+ 16us) server_negotiation.cc:299] Negotiation successful
0806 13:09:46.992377 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:09:46.992650 5075 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:37818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:09:46.994258 5075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:09:46.996297 5264 tablet_service.cc:1467] Processing CreateTablet for tablet bef509a241004557bb837ffe3f99a757 (DEFAULT_TABLE table=test-table [id=9b45affb961442d5b1560d6232e808bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:46.996502 5264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bef509a241004557bb837ffe3f99a757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:46.997814 5203 tablet_bootstrap.cc:492] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939: Bootstrap starting.
I20260806 13:09:46.998167 5203 tablet_bootstrap.cc:654] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:46.998660 5203 tablet_bootstrap.cc:492] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939: No bootstrap required, opened a new log
I20260806 13:09:46.998701 5203 ts_tablet_manager.cc:1397] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 13:09:46.998828 5203 raft_consensus.cc:348] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:46.998903 5203 raft_consensus.cc:374] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:46.998953 5203 raft_consensus.cc:729] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5304c57dea24bb1aeb60411618aa939, State: Initialized, Role: FOLLOWER
I20260806 13:09:46.998996 5203 consensus_queue.cc:260] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:46.999081 5203 ts_tablet_manager.cc:1428] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 13:09:47.000933 5765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.995980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50510 (local address 127.13.224.66:45183)
0806 13:09:46.996217 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:46.996222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:46.996233 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:09:46.996273 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:46.996275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:46.996290 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:46.996390 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:46.996476 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.997722 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.999925 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.000606 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.000666 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:47.000724 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:47.000822 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 13:09:47.000857 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":115,"threads_started":1}
I20260806 13:09:47.000931 5768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.996167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45030 (local address 127.13.224.67:43151)
0806 13:09:46.996327 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:46.996331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:46.996341 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:09:46.996371 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:46.996373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:46.996387 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:46.996462 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:46.996660 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.997715 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.999905 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.000589 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.000666 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:47.000723 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:47.000821 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 13:09:47.000857 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260806 13:09:47.000934 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.995852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:45183 (local address 127.0.0.1:50510)
0806 13:09:46.995879 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:46.995888 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:46.995927 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:46.996389 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:46.996390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:46.996392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:46.996428 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:46.996430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.998083 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.998084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:46.998963 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:46.998966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.000667 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.000669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:47.000741 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:47.000746 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:09:47.000807 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 13:09:47.000850 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":4}
I20260806 13:09:47.000936 5764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:46.995948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:43151 (local address 127.0.0.1:45030)
0806 13:09:46.996065 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:46.996074 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:46.996111 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:46.996540 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:46.996541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:46.996543 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:46.996583 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:46.996585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:46.999040 (+ 2455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:46.999040 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:46.999832 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:46.999834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.000669 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.000669 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:47.000741 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:47.000755 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 13:09:47.000807 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 13:09:47.000848 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":12,"thread_start_us":72,"threads_started":1}
I20260806 13:09:47.001562 5665 tablet_service.cc:1467] Processing CreateTablet for tablet bef509a241004557bb837ffe3f99a757 (DEFAULT_TABLE table=test-table [id=9b45affb961442d5b1560d6232e808bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:47.001667 5475 tablet_service.cc:1467] Processing CreateTablet for tablet bef509a241004557bb837ffe3f99a757 (DEFAULT_TABLE table=test-table [id=9b45affb961442d5b1560d6232e808bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:09:47.001842 5665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bef509a241004557bb837ffe3f99a757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:47.001999 5475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bef509a241004557bb837ffe3f99a757. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:09:47.004364 5770 tablet_bootstrap.cc:492] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768: Bootstrap starting.
I20260806 13:09:47.004464 5771 tablet_bootstrap.cc:492] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c: Bootstrap starting.
I20260806 13:09:47.004993 5771 tablet_bootstrap.cc:654] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:47.005054 5770 tablet_bootstrap.cc:654] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768: Neither blocks nor log segments found. Creating new log.
I20260806 13:09:47.005474 5771 log.cc:824] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:47.005501 5770 log.cc:824] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768: Log is configured to *not* fsync() on all Append() calls
I20260806 13:09:47.006177 5771 tablet_bootstrap.cc:492] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c: No bootstrap required, opened a new log
I20260806 13:09:47.006172 5770 tablet_bootstrap.cc:492] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768: No bootstrap required, opened a new log
I20260806 13:09:47.006232 5770 ts_tablet_manager.cc:1397] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:09:47.006237 5771 ts_tablet_manager.cc:1397] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:09:47.007486 5770 raft_consensus.cc:348] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:47.007507 5771 raft_consensus.cc:348] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:47.007613 5770 raft_consensus.cc:374] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:47.007622 5771 raft_consensus.cc:374] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:09:47.007651 5771 raft_consensus.cc:729] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a6ca56a4470464aafdbe60f10c7ce3c, State: Initialized, Role: FOLLOWER
I20260806 13:09:47.007634 5770 raft_consensus.cc:729] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bb03195a8847dda30e3f8ef6ee1768, State: Initialized, Role: FOLLOWER
I20260806 13:09:47.007786 5771 consensus_queue.cc:260] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:47.007784 5770 consensus_queue.cc:260] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:47.008056 5771 ts_tablet_manager.cc:1428] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:09:47.008061 5770 ts_tablet_manager.cc:1428] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:09:47.008170 5758 heartbeater.cc:503] Master 127.13.224.126:35327 was elected leader, sending a full tablet report...
I20260806 13:09:47.008203 5570 heartbeater.cc:503] Master 127.13.224.126:35327 was elected leader, sending a full tablet report...
I20260806 13:09:47.083158 5779 raft_consensus.cc:482] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:09:47.083267 5779 raft_consensus.cc:504] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:47.083612 5779 leader_election.cc:288] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151), a5304c57dea24bb1aeb60411618aa939 (127.13.224.65:35807)
I20260806 13:09:47.086315 5768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:47.083841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:60129 (local address 127.13.224.67:43151)
0806 13:09:47.083966 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:47.083969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:47.083978 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:09:47.084004 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:47.084005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:47.084014 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:47.084090 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:47.084252 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.084705 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.085593 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.085967 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.086064 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:47.086075 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:47.086224 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 13:09:47.086259 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 13:09:47.086311 5572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:47.083798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:43151 (local address 127.13.224.66:60129)
0806 13:09:47.083842 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:47.083854 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:47.083918 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:47.084100 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:47.084101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:47.084103 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:47.084172 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:47.084174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.084777 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.084778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:47.085517 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:47.085519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.086068 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.086068 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:47.086146 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:47.086149 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:09:47.086193 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 13:09:47.086233 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 13:09:47.086694 5783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:47.083900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:35807 (local address 127.13.224.66:59323)
0806 13:09:47.084065 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:47.084073 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:09:47.084111 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:09:47.084332 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:09:47.084333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:09:47.084335 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:09:47.084404 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:47.084405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.084978 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.084979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:47.085779 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:09:47.085781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.086428 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.086429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:09:47.086510 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:47.086514 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:09:47.086581 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 13:09:47.086615 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260806 13:09:47.086647 5693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bef509a241004557bb837ffe3f99a757" candidate_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bb03195a8847dda30e3f8ef6ee1768" is_pre_election: true
I20260806 13:09:47.086706 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:47.084170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:59323 (local address 127.13.224.65:35807)
0806 13:09:47.084224 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:47.084226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:47.084234 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:09:47.084258 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:47.084258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:47.084266 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:47.084317 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:09:47.084464 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.084912 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.085857 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.086321 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.086382 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:47.086391 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:47.086599 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 13:09:47.086647 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 13:09:47.086792 5693 raft_consensus.cc:2454] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a6ca56a4470464aafdbe60f10c7ce3c in term 0.
I20260806 13:09:47.087018 5409 leader_election.cc:302] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a6ca56a4470464aafdbe60f10c7ce3c, 78bb03195a8847dda30e3f8ef6ee1768; no voters:
I20260806 13:09:47.087131 5779 raft_consensus.cc:2781] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:09:47.087091 5292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bef509a241004557bb837ffe3f99a757" candidate_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5304c57dea24bb1aeb60411618aa939" is_pre_election: true
I20260806 13:09:47.087180 5779 raft_consensus.cc:482] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:09:47.087195 5779 raft_consensus.cc:3037] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:47.087242 5292 raft_consensus.cc:2454] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a6ca56a4470464aafdbe60f10c7ce3c in term 0.
I20260806 13:09:47.087914 5779 raft_consensus.cc:504] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:47.088019 5779 leader_election.cc:288] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [CANDIDATE]: Term 1 election: Requested vote from peers 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151), a5304c57dea24bb1aeb60411618aa939 (127.13.224.65:35807)
I20260806 13:09:47.088162 5693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bef509a241004557bb837ffe3f99a757" candidate_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78bb03195a8847dda30e3f8ef6ee1768"
I20260806 13:09:47.088171 5292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bef509a241004557bb837ffe3f99a757" candidate_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5304c57dea24bb1aeb60411618aa939"
I20260806 13:09:47.088230 5693 raft_consensus.cc:3037] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:47.088261 5292 raft_consensus.cc:3037] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:09:47.088898 5693 raft_consensus.cc:2454] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a6ca56a4470464aafdbe60f10c7ce3c in term 1.
I20260806 13:09:47.089052 5409 leader_election.cc:302] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a6ca56a4470464aafdbe60f10c7ce3c, 78bb03195a8847dda30e3f8ef6ee1768; no voters:
I20260806 13:09:47.089157 5779 raft_consensus.cc:2781] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:09:47.089311 5779 raft_consensus.cc:686] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [term 1 LEADER]: Becoming Leader. State: Replica: 4a6ca56a4470464aafdbe60f10c7ce3c, State: Running, Role: LEADER
I20260806 13:09:47.089325 5292 raft_consensus.cc:2454] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a6ca56a4470464aafdbe60f10c7ce3c in term 1.
I20260806 13:09:47.089406 5779 consensus_queue.cc:237] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:09:47.090159 5075 catalog_manager.cc:5495] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66). New cstate: current_term: 1 leader_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } health_report { overall_health: UNKNOWN } } }
W20260806 13:09:47.097410 5571 tablet.cc:2369] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:09:47.113384 5765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:47.111208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50522 (local address 127.13.224.66:45183)
0806 13:09:47.111263 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:47.111265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:47.111274 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:09:47.111304 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:47.111305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:47.111320 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:47.111374 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:09:47.111455 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.111887 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.112573 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.112866 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.112919 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:47.113096 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:09:47.113149 (+ 53us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 13:09:47.113297 (+ 148us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50522: token expired
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 13:09:47.113438 5765 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50522: token expired
I20260806 13:09:47.118934 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:47.113756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37824 (local address 127.13.224.126:35327)
0806 13:09:47.113801 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:47.113803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:47.113810 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:09:47.113830 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:47.113831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:47.113838 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:47.113883 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:09:47.114399 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.114868 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.115520 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.115708 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.115767 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:47.116798 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:09:47.116800 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:09:47.116800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:09:47.116807 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:09:47.117850 (+ 1043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:47.118220 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:47.118221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:47.118221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:47.118276 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:09:47.118440 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:09:47.118440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:09:47.118441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:09:47.118567 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:09:47.118658 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:47.118768 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 13:09:47.118847 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:09:47.121366 5765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:47.119669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50538 (local address 127.13.224.66:45183)
0806 13:09:47.119720 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 13:09:47.119721 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:09:47.119729 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:09:47.119746 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:09:47.119747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:09:47.119754 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:09:47.119791 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:09:47.119863 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.120280 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.120854 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:09:47.121015 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:09:47.121054 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:09:47.121131 (+ 77us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:09:47.121210 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:09:47.121256 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 13:09:47.121290 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 13:09:47.121305 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 13:09:47.123008 5292 raft_consensus.cc:1261] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Refusing update from remote peer 4a6ca56a4470464aafdbe60f10c7ce3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:09:47.123021 5693 raft_consensus.cc:1261] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768 [term 1 FOLLOWER]: Refusing update from remote peer 4a6ca56a4470464aafdbe60f10c7ce3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:09:47.123306 5779 consensus_queue.cc:1035] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:09:47.123448 5785 consensus_queue.cc:1035] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:09:47.125960 5789 mvcc.cc:205] Tried to move back new op lower bound from 7315545240053485568 to 7315545239918895104. Current Snapshot: MvccSnapshot[applied={T|T < 7315545240053485568}]
I20260806 13:09:47.126118 5788 mvcc.cc:205] Tried to move back new op lower bound from 7315545240053485568 to 7315545239918895104. Current Snapshot: MvccSnapshot[applied={T|T < 7315545240053485568}]
I20260806 13:09:47.126236 5790 mvcc.cc:205] Tried to move back new op lower bound from 7315545240053485568 to 7315545239918895104. Current Snapshot: MvccSnapshot[applied={T|T < 7315545240053485568 or (T in {7315545240053485568})}]
I20260806 13:09:47.128197 5074 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:09:47.128274 5074 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:09:47.128933 5074 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Sending DeleteTablet for 3 replicas of tablet bef509a241004557bb837ffe3f99a757
I20260806 13:09:47.129205 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 5133
I20260806 13:09:47.131556 5475 tablet_service.cc:1514] Processing DeleteTablet for tablet bef509a241004557bb837ffe3f99a757 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:47 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:50510
I20260806 13:09:47.132965 5665 tablet_service.cc:1514] Processing DeleteTablet for tablet bef509a241004557bb837ffe3f99a757 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:09:47 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:45030
I20260806 13:09:47.133014 5799 tablet_replica.cc:330] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c: stopping tablet replica
I20260806 13:09:47.133210 5799 raft_consensus.cc:2229] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:09:47.133391 5799 raft_consensus.cc:2258] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:09:47.135318 5799 ts_tablet_manager.cc:1905] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:09:47.137030 5799 ts_tablet_manager.cc:1918] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:09:47.137156 5799 log.cc:1196] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/bef509a241004557bb837ffe3f99a757
I20260806 13:09:47.137513 5799 ts_tablet_manager.cc:1939] T bef509a241004557bb837ffe3f99a757 P 4a6ca56a4470464aafdbe60f10c7ce3c: Deleting consensus metadata
I20260806 13:09:47.137948 5056 catalog_manager.cc:4841] TS 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183): tablet bef509a241004557bb837ffe3f99a757 (table test-table [id=9b45affb961442d5b1560d6232e808bd]) successfully deleted
I20260806 13:09:47.138365 5801 tablet_replica.cc:330] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768: stopping tablet replica
I20260806 13:09:47.138500 5801 raft_consensus.cc:2229] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:09:47.138646 5801 raft_consensus.cc:2258] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:09:47.139914 5801 ts_tablet_manager.cc:1905] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:09:47.141418 5801 ts_tablet_manager.cc:1918] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:09:47.141491 5801 log.cc:1196] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/bef509a241004557bb837ffe3f99a757
I20260806 13:09:47.141755 5801 ts_tablet_manager.cc:1939] T bef509a241004557bb837ffe3f99a757 P 78bb03195a8847dda30e3f8ef6ee1768: Deleting consensus metadata
I20260806 13:09:47.142477 5056 catalog_manager.cc:4841] TS 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151): tablet bef509a241004557bb837ffe3f99a757 (table test-table [id=9b45affb961442d5b1560d6232e808bd]) successfully deleted
W20260806 13:09:47.151935 5056 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:35807 (error 108)
I20260806 13:09:47.152231 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 5371
I20260806 13:09:47.153491 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:09:47.152291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:35807 (local address 127.0.0.1:41942)
0806 13:09:47.153414 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0806 13:09:47.153460 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:35807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1098}
W20260806 13:09:47.154850 5056 catalog_manager.cc:4584] TS a5304c57dea24bb1aeb60411618aa939 (127.13.224.65:35807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bef509a241004557bb837ffe3f99a757: Network error: Client connection negotiation failed: client connection to 127.13.224.65:35807: connect: Connection refused (error 111)
I20260806 13:09:47.163811 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 5573
I20260806 13:09:47.171692 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 5029
I20260806 13:10:07.180061 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35327
--webserver_interface=127.13.224.126
--webserver_port=36351
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:35327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:07.260231 5803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:07.260421 5803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:07.260440 5803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:07.261512 5803 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 13:10:07.261546 5803 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 13:10:07.261559 5803 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 13:10:07.261569 5803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:07.261580 5803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:07.261592 5803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:07.261602 5803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:07.261613 5803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:07.263017 5803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:35327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=36351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:07.263196 5803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:07.263368 5803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:07.267985 5809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:07.267975 5811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:07.268031 5803 server_base.cc:1034] running on GCE node
W20260806 13:10:07.267971 5808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:07.268473 5803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:07.268710 5803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:07.269845 5803 hybrid_clock.cc:648] HybridClock initialized: now 1786021807269834 us; error 28 us; skew 500 ppm
Aug 06 13:10:07 dist-test-slave-q9h9 krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021807, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:07.272091 5803 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:10:07.272490 5803 webserver.cc:460] Webserver started at http://127.13.224.126:36351/ using document root <none> and password file <none>
I20260806 13:10:07.272691 5803 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:07.272733 5803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:07.273900 5803 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:10:07.274497 5819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:07.274647 5803 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:07.274729 5803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "c69a62d1305d4da182f51503bfb8ca67"
format_stamp: "Formatted at 2026-08-06 13:09:25 on dist-test-slave-q9h9"
server_key: "81f6d1d768b084bce8b408c2166e9ae1"
server_key_iv: "53604aad607dff6cee5b3a6218595596"
server_key_version: "encryptionkey@0"
I20260806 13:10:07.275002 5803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:07.285681 5803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:07.287216 5803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:07.287314 5803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:07.290877 5803 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:35327
I20260806 13:10:07.290907 5893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:35327 every 8 connection(s)
I20260806 13:10:07.291220 5803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:07.292505 5895 sys_catalog.cc:263] Verifying existing consensus state
I20260806 13:10:07.293085 5895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Bootstrap starting.
I20260806 13:10:07.295498 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 5803
I20260806 13:10:07.295758 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:35807
--local_ip_for_outbound_sockets=127.13.224.65
--tserver_master_addrs=127.13.224.126:35327
--webserver_port=35579
--webserver_interface=127.13.224.65
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:10:07.301231 5895 log.cc:824] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:07.303324 5895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:10:07.303571 5895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Bootstrap complete.
I20260806 13:10:07.305508 5895 raft_consensus.cc:348] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } }
I20260806 13:10:07.305714 5895 raft_consensus.cc:729] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c69a62d1305d4da182f51503bfb8ca67, State: Initialized, Role: FOLLOWER
I20260806 13:10:07.305851 5895 consensus_queue.cc:260] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } }
I20260806 13:10:07.305924 5895 raft_consensus.cc:388] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:07.305974 5895 raft_consensus.cc:482] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:07.306037 5895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 2 FOLLOWER]: Advancing to term 3
I20260806 13:10:07.306945 5895 raft_consensus.cc:504] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } }
I20260806 13:10:07.307037 5895 leader_election.cc:302] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c69a62d1305d4da182f51503bfb8ca67; no voters:
I20260806 13:10:07.307193 5895 leader_election.cc:288] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 13:10:07.307240 5899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 13:10:07.307358 5899 raft_consensus.cc:686] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [term 3 LEADER]: Becoming Leader. State: Replica: c69a62d1305d4da182f51503bfb8ca67, State: Running, Role: LEADER
I20260806 13:10:07.307449 5895 sys_catalog.cc:564] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:07.307446 5899 consensus_queue.cc:237] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } }
I20260806 13:10:07.307720 5901 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c69a62d1305d4da182f51503bfb8ca67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } } }
I20260806 13:10:07.307726 5903 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c69a62d1305d4da182f51503bfb8ca67. Latest consensus state: current_term: 3 leader_uuid: "c69a62d1305d4da182f51503bfb8ca67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69a62d1305d4da182f51503bfb8ca67" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35327 } } }
I20260806 13:10:07.307812 5901 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:07.307821 5903 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:07.308013 5908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:07.308804 5908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:07.308921 5908 catalog_manager.cc:1259] Loaded cluster ID: 9dc4ff9a21a344d4925be17c81aaa3ff
I20260806 13:10:07.309134 5908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:07.309962 5908 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:07.310149 5908 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Loaded TSK: 0
I20260806 13:10:07.310187 5908 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c69a62d1305d4da182f51503bfb8ca67: Loaded TSK: 1
F20260806 13:10:07.310387 5908 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1a3c5e19ad google::LogMessage::Fail() at ??:0
@ 0x7f1a3c5e488a google::LogMessage::SendToLog() at ??:0
@ 0x7f1a3c5e14de google::LogMessage::Flush() at ??:0
@ 0x7f1a3c5e2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1a3ed4e104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f1a3ed4bd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f1a4453f1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f1a4453f48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f1a4453f594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f1a44505c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f1a4450fad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1a3e4af0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1a3e4a4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1a3f3086db start_thread at ??:0
@ 0x7f1a3b0df61f clone at ??:0
W20260806 13:10:07.393921 5897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:07.394134 5897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:07.394160 5897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:07.395262 5897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:07.395298 5897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:07.395323 5897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:10:07.396730 5897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:35807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=35579
--tserver_master_addrs=127.13.224.126:35327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:07.396934 5897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:07.397133 5897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:07.401729 5959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:07.401813 5961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:07.401755 5958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:07.402098 5897 server_base.cc:1034] running on GCE node
I20260806 13:10:07.402275 5897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:07.402478 5897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:07.403614 5897 hybrid_clock.cc:648] HybridClock initialized: now 1786021807403600 us; error 34 us; skew 500 ppm
Aug 06 13:10:07 dist-test-slave-q9h9 krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021807, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:07.406068 5897 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:10:07.406481 5897 webserver.cc:460] Webserver started at http://127.13.224.65:35579/ using document root <none> and password file <none>
I20260806 13:10:07.406677 5897 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:07.406724 5897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:07.407963 5897 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:10:07.408620 5970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:07.408771 5897 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:07.408849 5897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "a5304c57dea24bb1aeb60411618aa939"
format_stamp: "Formatted at 2026-08-06 13:09:25 on dist-test-slave-q9h9"
server_key: "ac4826dff9c8fb7d4a38640fc256e36e"
server_key_iv: "69ca0e434bd9f4d820559f5910d7117f"
server_key_version: "encryptionkey@0"
I20260806 13:10:07.409138 5897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:07.442852 5897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:07.444885 5897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:07.445020 5897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:07.445243 5897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:07.445665 5980 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 13:10:07.447638 5897 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 13:10:07.447686 5897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 13:10:07.447718 5897 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 13:10:07.448333 5897 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 13:10:07.448365 5897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 13:10:07.448465 5980 tablet_bootstrap.cc:492] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939: Bootstrap starting.
I20260806 13:10:07.454977 5897 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:35807
I20260806 13:10:07.455101 6144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:35807 every 8 connection(s)
I20260806 13:10:07.455375 5897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:10:07.459570 5980 log.cc:824] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:07.461403 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 5897
I20260806 13:10:07.461671 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:45183
--local_ip_for_outbound_sockets=127.13.224.66
--tserver_master_addrs=127.13.224.126:35327
--webserver_port=35309
--webserver_interface=127.13.224.66
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:10:07.462178 5980 tablet_bootstrap.cc:492] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 13:10:07.462483 5980 tablet_bootstrap.cc:492] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939: Bootstrap complete.
I20260806 13:10:07.463661 5980 ts_tablet_manager.cc:1397] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.007s
I20260806 13:10:07.465023 5980 raft_consensus.cc:348] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:10:07.465363 5980 raft_consensus.cc:729] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5304c57dea24bb1aeb60411618aa939, State: Initialized, Role: FOLLOWER
I20260806 13:10:07.465524 5980 consensus_queue.cc:260] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:10:07.465793 5980 ts_tablet_manager.cc:1428] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260806 13:10:07.495393 6149 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:07.456143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:43383)
0806 13:10:07.456312 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:07.456353 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:07.456761 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:07.495196 (+ 38435us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
W20260806 13:10:07.495652 6145 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:35327 (0 consecutive failures): Network error: Failed to ping master at 127.13.224.126:35327: Client connection negotiation failed: client connection to 127.13.224.126:35327: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 13:10:07.496004 6149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:07.495888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:41961)
0806 13:10:07.495939 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:07.495981 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 13:10:07.496306 6149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:07.496234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:36175)
0806 13:10:07.496267 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:07.496289 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 13:10:07.496400 6145 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 13:10:07.544013 6154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:07.544205 6154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:07.544227 6154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:07.545305 6154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:07.545338 6154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:07.545362 6154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:10:07.546772 6154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:45183
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=35309
--tserver_master_addrs=127.13.224.126:35327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:07.546957 6154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:07.547137 6154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:07.551790 6162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:07.551800 6163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:07.551955 6154 server_base.cc:1034] running on GCE node
W20260806 13:10:07.551860 6165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:07.552259 6154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:07.552493 6154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:07.553633 6154 hybrid_clock.cc:648] HybridClock initialized: now 1786021807553619 us; error 26 us; skew 500 ppm
Aug 06 13:10:07 dist-test-slave-q9h9 krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021807, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:07.555974 6154 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:10:07.556380 6154 webserver.cc:460] Webserver started at http://127.13.224.66:35309/ using document root <none> and password file <none>
I20260806 13:10:07.556597 6154 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:07.556641 6154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:07.557832 6154 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:10:07.558393 6177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:07.558549 6154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:07.558640 6154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "4a6ca56a4470464aafdbe60f10c7ce3c"
format_stamp: "Formatted at 2026-08-06 13:09:26 on dist-test-slave-q9h9"
server_key: "a1ab59117fb9c0251551ede82c3e3b4f"
server_key_iv: "ac14295377b9e038f392e5f9914828d2"
server_key_version: "encryptionkey@0"
I20260806 13:10:07.558947 6154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:07.564958 6154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:07.566711 6154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:07.566825 6154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:07.567010 6154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:07.567260 6154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:07.567293 6154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:07.567315 6154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:07.567334 6154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:07.573319 6154 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:45183
I20260806 13:10:07.573376 6344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:45183 every 8 connection(s)
I20260806 13:10:07.573681 6154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:10:07.574682 6351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:07.574322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:48909)
0806 13:10:07.574473 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:07.574543 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
W20260806 13:10:07.575059 6348 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:35327 (0 consecutive failures): Network error: Failed to ping master at 127.13.224.126:35327: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
I20260806 13:10:07.575320 6351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:07.575225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:49755)
0806 13:10:07.575267 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:07.575295 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 13:10:07.575596 6351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:07.575517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:45271)
0806 13:10:07.575552 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:07.575575 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 13:10:07.575695 6348 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 13:10:07.576767 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 6154
I20260806 13:10:07.577010 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:43151
--local_ip_for_outbound_sockets=127.13.224.67
--tserver_master_addrs=127.13.224.126:35327
--webserver_port=33819
--webserver_interface=127.13.224.67
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:34887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:07.658665 6352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:07.658855 6352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:07.658885 6352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:07.659976 6352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:07.660013 6352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:07.660048 6352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:10:07.661531 6352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:34887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:43151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=33819
--tserver_master_addrs=127.13.224.126:35327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:07.661767 6352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:07.661964 6352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:07.666693 6362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:07.666702 6359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:07.666708 6360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:07.666853 6352 server_base.cc:1034] running on GCE node
I20260806 13:10:07.667097 6352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:07.667322 6352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:07.668454 6352 hybrid_clock.cc:648] HybridClock initialized: now 1786021807668443 us; error 33 us; skew 500 ppm
Aug 06 13:10:07 dist-test-slave-q9h9 krb5kdc[4214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021807, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:07.674839 6352 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:10:07.675227 6352 webserver.cc:460] Webserver started at http://127.13.224.67:33819/ using document root <none> and password file <none>
I20260806 13:10:07.675441 6352 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:07.675483 6352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:07.676702 6352 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:10:07.677255 6374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:07.677441 6352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:07.677522 6352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "78bb03195a8847dda30e3f8ef6ee1768"
format_stamp: "Formatted at 2026-08-06 13:09:26 on dist-test-slave-q9h9"
server_key: "8e94ab67baca76e358f0f725d930840c"
server_key_iv: "578951dfe00a0fb00259d9afff6130b3"
server_key_version: "encryptionkey@0"
I20260806 13:10:07.677810 6352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:07.688549 6352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:07.690092 6352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:07.690213 6352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:07.690392 6352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:07.690657 6352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:07.690690 6352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:07.690717 6352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:07.690742 6352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:07.696168 6352 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:43151
I20260806 13:10:07.696228 6527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:43151 every 8 connection(s)
I20260806 13:10:07.696506 6352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:10:07.697408 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:07.697087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:55877)
0806 13:10:07.697214 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:07.697282 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":63,"threads_started":1}
W20260806 13:10:07.697796 6528 heartbeater.cc:644] Failed to heartbeat to 127.13.224.126:35327 (0 consecutive failures): Network error: Failed to ping master at 127.13.224.126:35327: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
I20260806 13:10:07.698055 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:07.697954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:60501)
0806 13:10:07.697991 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:07.698032 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 13:10:07.698304 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:07.698234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:45981)
0806 13:10:07.698271 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:07.698290 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 13:10:07.698408 6528 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 13:10:07.702142 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 6352
I20260806 13:10:08.497020 6533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:08.496762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:46021)
0806 13:10:08.496915 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:08.496954 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260806 13:10:08.576192 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:08.575979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:33739)
0806 13:10:08.576108 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:08.576144 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":55,"threads_started":1}
I20260806 13:10:08.698885 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:08.698688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:34229)
0806 13:10:08.698803 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:08.698837 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260806 13:10:09.204403 6536 raft_consensus.cc:482] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:10:09.204517 6536 raft_consensus.cc:504] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:10:09.204859 6536 leader_election.cc:288] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183), 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151)
Aug 06 13:10:09 dist-test-slave-q9h9 krb5kdc[4214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021807, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.66@KRBTEST.COM
Aug 06 13:10:09 dist-test-slave-q9h9 krb5kdc[4214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021807, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.67@KRBTEST.COM
I20260806 13:10:09.215603 6542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:09.205200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:43151 (local address 127.13.224.65:35349)
0806 13:10:09.205301 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:09.205310 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:09.205346 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:09.206219 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:09.206221 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:09.206496 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:09.206687 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:09.206693 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:09.207829 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:09.207830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:09.208571 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:09.208573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:09.209149 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:09.209150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:09.209270 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:09.209816 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:09.209821 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:09.211643 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:09.213602 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:09.213603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:09.213603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:09.214057 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:09.214344 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:09.214345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:09.214345 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:09.214517 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:09.215015 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:09.215025 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:09.215188 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 13:10:09.215303 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 13:10:09.215478 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":42,"thread_start_us":59,"threads_started":1}
I20260806 13:10:09.215682 6541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:09.205102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:45263 (local address 127.13.224.66:45183)
0806 13:10:09.205237 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:09.205249 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:09.205277 (+ 28us) server_negotiation.cc:407] Connection header received
0806 13:10:09.205826 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:09.205830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:09.205876 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:09.205996 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:09.206810 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:09.207654 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:09.208660 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:09.209061 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:09.209162 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:09.211442 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:09.211447 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:09.211447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:09.211459 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:09.213481 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:09.214162 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:09.214165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:09.214166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:09.214276 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:09.214591 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:09.214592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:09.214592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:09.214913 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:09.215004 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:09.215417 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 13:10:09.215564 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":43,"threads_started":1}
I20260806 13:10:09.215698 6544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:09.205337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:35349 (local address 127.13.224.67:43151)
0806 13:10:09.205510 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:09.205521 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:09.205542 (+ 21us) server_negotiation.cc:407] Connection header received
0806 13:10:09.206008 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:09.206011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:09.206085 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:09.206219 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:09.206797 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:09.207737 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:09.208679 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:09.209061 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:09.209163 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:09.211742 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:09.211746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:09.211747 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:09.211757 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:09.213525 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:09.214158 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:09.214162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:09.214162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:09.214276 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:09.214581 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:09.214582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:09.214582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:09.214913 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:09.215004 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:09.215414 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 13:10:09.215564 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260806 13:10:09.215660 6539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:09.204997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:45183 (local address 127.13.224.65:45263)
0806 13:10:09.205148 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:09.205160 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:09.205219 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:09.206089 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:09.206092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:09.206495 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:09.206687 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:09.206694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:09.207748 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:09.207749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:09.208573 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:09.208576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:09.209157 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:09.209159 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:09.209269 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:09.209779 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:09.209788 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:09.211313 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:09.213580 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:09.213590 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:09.213602 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:09.214057 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:09.214344 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:09.214345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:09.214345 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:09.214517 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:09.215017 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:09.215025 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:09.215188 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 13:10:09.215302 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 13:10:09.215545 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":160,"thread_start_us":75,"threads_started":1}
W20260806 13:10:09.216836 5973 leader_election.cc:341] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183): Not found: Tablet not found: bef509a241004557bb837ffe3f99a757
W20260806 13:10:09.216915 5973 leader_election.cc:341] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151): Not found: Tablet not found: bef509a241004557bb837ffe3f99a757
I20260806 13:10:09.216943 5973 leader_election.cc:302] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5304c57dea24bb1aeb60411618aa939; no voters: 4a6ca56a4470464aafdbe60f10c7ce3c, 78bb03195a8847dda30e3f8ef6ee1768
I20260806 13:10:09.217063 6536 raft_consensus.cc:2726] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:10:09.497543 6539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:09.497418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:59387)
0806 13:10:09.497476 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:09.497502 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 13:10:09.576828 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:09.576618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:54805)
0806 13:10:09.576743 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:09.576784 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260806 13:10:09.699493 6549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:09.699288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:35039)
0806 13:10:09.699413 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:09.699454 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260806 13:10:10.498168 6550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:10.497943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:47211)
0806 13:10:10.498084 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:10.498120 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260806 13:10:10.577472 6552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:10.577256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:40597)
0806 13:10:10.577389 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:10.577430 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260806 13:10:10.700057 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:10.699866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:44617)
0806 13:10:10.699984 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:10.700017 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
I20260806 13:10:10.949970 6554 raft_consensus.cc:482] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:10:10.950088 6554 raft_consensus.cc:504] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:10:10.950258 6554 leader_election.cc:288] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183), 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151)
W20260806 13:10:10.950656 5973 leader_election.cc:341] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183): Not found: Tablet not found: bef509a241004557bb837ffe3f99a757
W20260806 13:10:10.950722 5973 leader_election.cc:341] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151): Not found: Tablet not found: bef509a241004557bb837ffe3f99a757
I20260806 13:10:10.950736 5973 leader_election.cc:302] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5304c57dea24bb1aeb60411618aa939; no voters: 4a6ca56a4470464aafdbe60f10c7ce3c, 78bb03195a8847dda30e3f8ef6ee1768
I20260806 13:10:10.950837 6554 raft_consensus.cc:2726] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:10:11.498816 6555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:11.498582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:33003)
0806 13:10:11.498721 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:11.498765 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260806 13:10:11.578030 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:11.577820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:40537)
0806 13:10:11.577943 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:11.577979 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":55,"threads_started":1}
I20260806 13:10:11.700626 6557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:11.700429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:40653)
0806 13:10:11.700544 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:11.700575 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
I20260806 13:10:12.499532 6558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:12.499304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:60013)
0806 13:10:12.499450 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:12.499485 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260806 13:10:12.578589 6559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:12.578391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:32989)
0806 13:10:12.578511 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:12.578547 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260806 13:10:12.701200 6561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:12.700992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:54299)
0806 13:10:12.701122 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:12.701159 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
I20260806 13:10:13.500170 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:13.499915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:36927)
0806 13:10:13.500080 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:13.500121 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260806 13:10:13.510578 6567 raft_consensus.cc:482] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:10:13.510658 6567 raft_consensus.cc:504] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:10:13.510922 6567 leader_election.cc:288] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183), 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151)
W20260806 13:10:13.511281 5973 leader_election.cc:341] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183): Not found: Tablet not found: bef509a241004557bb837ffe3f99a757
W20260806 13:10:13.511343 5973 leader_election.cc:341] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151): Not found: Tablet not found: bef509a241004557bb837ffe3f99a757
I20260806 13:10:13.511356 5973 leader_election.cc:302] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5304c57dea24bb1aeb60411618aa939; no voters: 4a6ca56a4470464aafdbe60f10c7ce3c, 78bb03195a8847dda30e3f8ef6ee1768
I20260806 13:10:13.511425 6567 raft_consensus.cc:2726] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:10:13.579136 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:13.578943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:50491)
0806 13:10:13.579062 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:13.579096 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":52,"threads_started":1}
I20260806 13:10:13.701807 6569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:13.701589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:35491)
0806 13:10:13.701716 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:13.701750 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260806 13:10:14.500845 6570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:14.500602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:46987)
0806 13:10:14.500734 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:14.500793 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260806 13:10:14.579715 6571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:14.579473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:45707)
0806 13:10:14.579606 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:14.579648 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":57,"threads_started":1}
I20260806 13:10:14.702425 6572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:14.702199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:50761)
0806 13:10:14.702344 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:14.702381 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260806 13:10:15.501518 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:15.501311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:46835)
0806 13:10:15.501438 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:15.501472 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260806 13:10:15.580228 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:15.580054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:37461)
0806 13:10:15.580158 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:15.580191 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":54,"threads_started":1}
I20260806 13:10:15.702967 6575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:15.702790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:57333)
0806 13:10:15.702895 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:15.702927 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":55,"threads_started":1}
I20260806 13:10:16.502146 6577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:16.501919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:44873)
0806 13:10:16.502067 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:16.502100 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":70,"threads_started":1}
I20260806 13:10:16.580782 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:16.580564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:44357)
0806 13:10:16.580682 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:16.580735 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
I20260806 13:10:16.678124 6579 raft_consensus.cc:482] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:10:16.678220 6579 raft_consensus.cc:504] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:10:16.678360 6579 leader_election.cc:288] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183), 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151)
W20260806 13:10:16.678694 5973 leader_election.cc:341] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151): Not found: Tablet not found: bef509a241004557bb837ffe3f99a757
W20260806 13:10:16.678782 5973 leader_election.cc:341] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183): Not found: Tablet not found: bef509a241004557bb837ffe3f99a757
I20260806 13:10:16.678814 5973 leader_election.cc:302] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5304c57dea24bb1aeb60411618aa939; no voters: 4a6ca56a4470464aafdbe60f10c7ce3c, 78bb03195a8847dda30e3f8ef6ee1768
I20260806 13:10:16.678900 6579 raft_consensus.cc:2726] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:10:16.703507 6582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:16.703318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:37179)
0806 13:10:16.703444 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:16.703471 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260806 13:10:17.502764 6583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:17.502557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:45473)
0806 13:10:17.502674 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:17.502708 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
I20260806 13:10:17.581317 6584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:17.581124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:38761)
0806 13:10:17.581239 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:17.581277 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
I20260806 13:10:17.704063 6585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:17.703874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:47133)
0806 13:10:17.703992 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:17.704025 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260806 13:10:18.503428 6587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:18.503195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:59945)
0806 13:10:18.503346 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:18.503380 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260806 13:10:18.581864 6588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:18.581662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:41555)
0806 13:10:18.581794 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:18.581827 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260806 13:10:18.704640 6589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:18.704456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:44015)
0806 13:10:18.704574 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:18.704604 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260806 13:10:19.504082 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:19.503853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:45543)
0806 13:10:19.503999 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:19.504037 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260806 13:10:19.582417 6593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:19.582235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:41909)
0806 13:10:19.582345 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:19.582377 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260806 13:10:19.705300 6594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:19.705071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:33935)
0806 13:10:19.705228 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:19.705260 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260806 13:10:19.910426 6595 raft_consensus.cc:482] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:10:19.910513 6595 raft_consensus.cc:504] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:10:19.910640 6595 leader_election.cc:288] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183), 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151)
W20260806 13:10:19.911018 5973 leader_election.cc:341] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183): Not found: Tablet not found: bef509a241004557bb837ffe3f99a757
W20260806 13:10:19.911082 5973 leader_election.cc:341] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151): Not found: Tablet not found: bef509a241004557bb837ffe3f99a757
I20260806 13:10:19.911094 5973 leader_election.cc:302] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5304c57dea24bb1aeb60411618aa939; no voters: 4a6ca56a4470464aafdbe60f10c7ce3c, 78bb03195a8847dda30e3f8ef6ee1768
I20260806 13:10:19.911165 6595 raft_consensus.cc:2726] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:10:20.504664 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:20.504474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:57495)
0806 13:10:20.504588 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:20.504622 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260806 13:10:20.582938 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:20.582741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:40943)
0806 13:10:20.582861 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:20.582898 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":55,"threads_started":1}
I20260806 13:10:20.705868 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:20.705687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:54443)
0806 13:10:20.705797 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:20.705827 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":66,"threads_started":1}
I20260806 13:10:21.505367 6601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:21.505117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:50445)
0806 13:10:21.505272 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:21.505317 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260806 13:10:21.583609 6602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:21.583395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:41141)
0806 13:10:21.583535 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:21.583573 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260806 13:10:21.706507 6603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:21.706269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:41041)
0806 13:10:21.706431 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:21.706467 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260806 13:10:21.765455 6605 raft_consensus.cc:482] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:10:21.765550 6605 raft_consensus.cc:504] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a6ca56a4470464aafdbe60f10c7ce3c" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 45183 } } peers { permanent_uuid: "78bb03195a8847dda30e3f8ef6ee1768" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 43151 } } peers { permanent_uuid: "a5304c57dea24bb1aeb60411618aa939" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 35807 } }
I20260806 13:10:21.765723 6605 leader_election.cc:288] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183), 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151)
W20260806 13:10:21.766137 5973 leader_election.cc:341] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78bb03195a8847dda30e3f8ef6ee1768 (127.13.224.67:43151): Not found: Tablet not found: bef509a241004557bb837ffe3f99a757
W20260806 13:10:21.766201 5973 leader_election.cc:341] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4a6ca56a4470464aafdbe60f10c7ce3c (127.13.224.66:45183): Not found: Tablet not found: bef509a241004557bb837ffe3f99a757
I20260806 13:10:21.766225 5973 leader_election.cc:302] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5304c57dea24bb1aeb60411618aa939; no voters: 4a6ca56a4470464aafdbe60f10c7ce3c, 78bb03195a8847dda30e3f8ef6ee1768
I20260806 13:10:21.766314 6605 raft_consensus.cc:2726] T bef509a241004557bb837ffe3f99a757 P a5304c57dea24bb1aeb60411618aa939 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 13:10:22.505985 6607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:22.505782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.65:53267)
0806 13:10:22.505912 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:22.505940 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260806 13:10:22.584200 6609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:22.584021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.66:45517)
0806 13:10:22.584133 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:22.584163 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
I20260806 13:10:22.707063 6610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:22.706892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:35327 (local address 127.13.224.67:36057)
0806 13:10:22.706993 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:22.707023 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.126:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":52,"threads_started":1}
I20260806 13:10:22.711054 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 5897
I20260806 13:10:22.719872 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 6154
I20260806 13:10:22.727860 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 6352
2026-08-06T13:10:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58265 ms)
[----------] 30 tests from SecurityITest (231238 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:22 dist-test-slave-q9h9 krb5kdc[6614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:22 dist-test-slave-q9h9 krb5kdc[6614](info): set up 2 sockets
Aug 06 13:10:22 dist-test-slave-q9h9 krb5kdc[6614](info): commencing operation
krb5kdc: starting...
W20260806 13:10:23.835279 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:23 dist-test-slave-q9h9 krb5kdc[6614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:10:23.940542 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:42001
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:46045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:42001
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:24.019753 6635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:24.019932 6635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:24.019953 6635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:24.021044 6635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:24.021083 6635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:24.021096 6635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:24.021121 6635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:24.021143 6635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:24.022563 6635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:42001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:42001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:24.022778 6635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:24.022971 6635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:24.027626 6644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:24.027634 6635 server_base.cc:1034] running on GCE node
W20260806 13:10:24.027649 6645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:24.027634 6648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:24.028014 6635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:24.028283 6635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:24.029418 6635 hybrid_clock.cc:648] HybridClock initialized: now 1786021824029404 us; error 29 us; skew 500 ppm
Aug 06 13:10:24 dist-test-slave-q9h9 krb5kdc[6614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021824, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:24.031682 6635 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:10:24.032079 6635 webserver.cc:460] Webserver started at http://127.13.224.126:43023/ using document root <none> and password file <none>
I20260806 13:10:24.032276 6635 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:24.032325 6635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:24.032445 6635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:24.033375 6635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "f3d3efb1b9be46a1a96d28699cbf3fb4"
format_stamp: "Formatted at 2026-08-06 13:10:24 on dist-test-slave-q9h9"
server_key: "aecec611fa32c156130b04694b13aee3"
server_key_iv: "bbe09da88ca3472e3a111c1c27d22bd8"
server_key_version: "encryptionkey@0"
I20260806 13:10:24.033684 6635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "f3d3efb1b9be46a1a96d28699cbf3fb4"
format_stamp: "Formatted at 2026-08-06 13:10:24 on dist-test-slave-q9h9"
server_key: "aecec611fa32c156130b04694b13aee3"
server_key_iv: "bbe09da88ca3472e3a111c1c27d22bd8"
server_key_version: "encryptionkey@0"
I20260806 13:10:24.034988 6635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:10:24.035666 6660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:24.035804 6635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:10:24.035887 6635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "f3d3efb1b9be46a1a96d28699cbf3fb4"
format_stamp: "Formatted at 2026-08-06 13:10:24 on dist-test-slave-q9h9"
server_key: "aecec611fa32c156130b04694b13aee3"
server_key_iv: "bbe09da88ca3472e3a111c1c27d22bd8"
server_key_version: "encryptionkey@0"
I20260806 13:10:24.035944 6635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:24.061038 6635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:24.062914 6635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:24.063038 6635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:24.066685 6635 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:42001
I20260806 13:10:24.066725 6738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:42001 every 8 connection(s)
I20260806 13:10:24.067013 6635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:24.067795 6739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:24.069746 6739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4: Bootstrap starting.
I20260806 13:10:24.070262 6739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:24.070479 6739 log.cc:824] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:24.071043 6739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4: No bootstrap required, opened a new log
I20260806 13:10:24.072475 6739 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3d3efb1b9be46a1a96d28699cbf3fb4" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42001 } }
I20260806 13:10:24.072598 6739 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:24.072628 6739 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d3efb1b9be46a1a96d28699cbf3fb4, State: Initialized, Role: FOLLOWER
I20260806 13:10:24.072736 6739 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3d3efb1b9be46a1a96d28699cbf3fb4" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42001 } }
I20260806 13:10:24.072811 6739 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:24.072844 6739 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:24.072875 6739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:24.073316 6739 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3d3efb1b9be46a1a96d28699cbf3fb4" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42001 } }
I20260806 13:10:24.073396 6739 leader_election.cc:302] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3d3efb1b9be46a1a96d28699cbf3fb4; no voters:
I20260806 13:10:24.073510 6739 leader_election.cc:288] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:24.073544 6742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:24.073696 6742 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [term 1 LEADER]: Becoming Leader. State: Replica: f3d3efb1b9be46a1a96d28699cbf3fb4, State: Running, Role: LEADER
I20260806 13:10:24.073719 6739 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:24.073793 6742 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3d3efb1b9be46a1a96d28699cbf3fb4" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42001 } }
I20260806 13:10:24.074157 6745 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3d3efb1b9be46a1a96d28699cbf3fb4. Latest consensus state: current_term: 1 leader_uuid: "f3d3efb1b9be46a1a96d28699cbf3fb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3d3efb1b9be46a1a96d28699cbf3fb4" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42001 } } }
I20260806 13:10:24.074259 6744 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3d3efb1b9be46a1a96d28699cbf3fb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3d3efb1b9be46a1a96d28699cbf3fb4" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42001 } } }
I20260806 13:10:24.074337 6745 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:24.074399 6744 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:24.074697 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 6635
I20260806 13:10:24.074736 6753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:24.074812 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:24.074988 14209 external_mini_cluster.cc:1428] Setting key 84e4ec3bd018eb7c39212e43613984c9
I20260806 13:10:24.075202 6753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:24.076741 6753 catalog_manager.cc:1347] Generated new cluster ID: b06c249173d44920827ce70a96745608
I20260806 13:10:24.076789 6753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:10:24 dist-test-slave-q9h9 krb5kdc[6614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:24.085986 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.075543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55608 (local address 127.13.224.126:42001)
0806 13:10:24.075920 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:24.075928 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:24.075958 (+ 30us) server_negotiation.cc:407] Connection header received
0806 13:10:24.076063 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:24.076070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:24.076120 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:24.076262 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:24.077027 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.078414 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.079427 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.080054 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.080166 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.082714 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:24.082718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:24.082718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:24.082730 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:24.084473 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:24.084864 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:24.084867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:24.084867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:24.084956 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:24.085141 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:24.085142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:24.085142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:24.085413 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:24.085539 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:24.085655 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 13:10:24.085770 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":73,"threads_started":1}
I20260806 13:10:24.100812 6753 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:24.101275 6753 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:24.108196 6753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4: Generated new TSK 0
I20260806 13:10:24.108377 6753 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:10:24.138149 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:42001
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:46045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:24.216688 6778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:24.216876 6778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:24.216897 6778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:24.217994 6778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:24.218061 6778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:24.218107 6778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:10:24.219512 6778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:42001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:24.219715 6778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:24.219918 6778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:24.224455 6790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:24.224468 6784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:24.224466 6785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:24.224714 6778 server_base.cc:1034] running on GCE node
I20260806 13:10:24.224877 6778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:24.225076 6778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:24.226238 6778 hybrid_clock.cc:648] HybridClock initialized: now 1786021824226227 us; error 26 us; skew 500 ppm
Aug 06 13:10:24 dist-test-slave-q9h9 krb5kdc[6614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021824, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:24.228435 6778 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:10:24.228816 6778 webserver.cc:460] Webserver started at http://127.13.224.65:41417/ using document root <none> and password file <none>
I20260806 13:10:24.228997 6778 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:24.229043 6778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:24.229162 6778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:24.230201 6778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "3647dacda7fe4c0fadc6a02472ca3b92"
format_stamp: "Formatted at 2026-08-06 13:10:24 on dist-test-slave-q9h9"
server_key: "c59bddc61e032b0d9f1ebe68492c876c"
server_key_iv: "37d543087b8f5a08690e1a7d67e8204d"
server_key_version: "encryptionkey@0"
I20260806 13:10:24.230509 6778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "3647dacda7fe4c0fadc6a02472ca3b92"
format_stamp: "Formatted at 2026-08-06 13:10:24 on dist-test-slave-q9h9"
server_key: "c59bddc61e032b0d9f1ebe68492c876c"
server_key_iv: "37d543087b8f5a08690e1a7d67e8204d"
server_key_version: "encryptionkey@0"
I20260806 13:10:24.231658 6778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:10:24.232276 6802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:24.232434 6778 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:24.232509 6778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "3647dacda7fe4c0fadc6a02472ca3b92"
format_stamp: "Formatted at 2026-08-06 13:10:24 on dist-test-slave-q9h9"
server_key: "c59bddc61e032b0d9f1ebe68492c876c"
server_key_iv: "37d543087b8f5a08690e1a7d67e8204d"
server_key_version: "encryptionkey@0"
I20260806 13:10:24.232570 6778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:24.240419 6778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:24.241911 6778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:24.242065 6778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:24.242271 6778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:24.242530 6778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:24.242563 6778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:24.242581 6778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:24.242604 6778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:24.247982 6778 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:41329
I20260806 13:10:24.248040 6961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:41329 every 8 connection(s)
I20260806 13:10:24.248288 6778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:10:24.252101 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 6778
I20260806 13:10:24.252211 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:10:24.252393 14209 external_mini_cluster.cc:1428] Setting key efb1f7ec34290127b53494426306ad46
Aug 06 13:10:24 dist-test-slave-q9h9 krb5kdc[6614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021824, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:24.257067 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.248866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:37579 (local address 127.13.224.126:42001)
0806 13:10:24.248908 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:24.248910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:24.249079 (+ 169us) server_negotiation.cc:407] Connection header received
0806 13:10:24.249503 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:24.249505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:24.249518 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:24.249576 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:24.250464 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.250959 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.251870 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.252159 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.252228 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.254446 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:24.254447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:24.254447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:24.254454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:24.255869 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:24.256276 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:24.256277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:24.256277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:24.256323 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:24.256541 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:24.256541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:24.256542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:24.256628 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:24.256672 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:24.256946 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 13:10:24.257007 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:10:24.257303 6966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.248842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:42001 (local address 127.13.224.65:37579)
0806 13:10:24.248979 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:24.249010 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:24.249413 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:24.249638 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:24.249642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:24.250201 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:24.250376 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.250380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.251037 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.251038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.251792 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.251794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.252251 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.252252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.252382 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.252832 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:24.252840 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:24.254361 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:24.255934 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:24.255942 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:24.255945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:24.256204 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:24.256363 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:24.256363 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:24.256364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:24.256489 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:24.256680 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:24.256685 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:24.256828 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 13:10:24.256934 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 13:10:24.257078 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260806 13:10:24.257815 6962 heartbeater.cc:347] Connected to a master server at 127.13.224.126:42001
I20260806 13:10:24.257908 6962 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:24.258114 6962 heartbeater.cc:511] Master 127.13.224.126:42001 requested a full tablet report, sending...
I20260806 13:10:24.258579 6690 ts_manager.cc:194] Registered new tserver with Master: 3647dacda7fe4c0fadc6a02472ca3b92 (127.13.224.65:41329)
I20260806 13:10:24.259153 6690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:37579
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:10:24.275835 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:42001
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:46045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:24.353984 6971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:24.354187 6971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:24.354216 6971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:24.355376 6971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:24.355463 6971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:24.355556 6971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:10:24.357173 6971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:42001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:24.357396 6971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:24.357597 6971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:24.362252 6980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:24.362291 6983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:24.362326 6971 server_base.cc:1034] running on GCE node
W20260806 13:10:24.362309 6979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:24.362622 6971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:24.362800 6971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:24.363932 6971 hybrid_clock.cc:648] HybridClock initialized: now 1786021824363921 us; error 30 us; skew 500 ppm
Aug 06 13:10:24 dist-test-slave-q9h9 krb5kdc[6614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021824, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:24.366114 6971 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:10:24.366495 6971 webserver.cc:460] Webserver started at http://127.13.224.66:40739/ using document root <none> and password file <none>
I20260806 13:10:24.366685 6971 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:24.366724 6971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:24.366809 6971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:24.367688 6971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "8fdfc54037af48619469828ad8a2454e"
format_stamp: "Formatted at 2026-08-06 13:10:24 on dist-test-slave-q9h9"
server_key: "80aab16ec08c67e92eb751d4118324ce"
server_key_iv: "fe3d76fb620ab4f00bc8194f2c775f7c"
server_key_version: "encryptionkey@0"
I20260806 13:10:24.367949 6971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "8fdfc54037af48619469828ad8a2454e"
format_stamp: "Formatted at 2026-08-06 13:10:24 on dist-test-slave-q9h9"
server_key: "80aab16ec08c67e92eb751d4118324ce"
server_key_iv: "fe3d76fb620ab4f00bc8194f2c775f7c"
server_key_version: "encryptionkey@0"
I20260806 13:10:24.369063 6971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:24.369675 6993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:24.369832 6971 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:24.369913 6971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "8fdfc54037af48619469828ad8a2454e"
format_stamp: "Formatted at 2026-08-06 13:10:24 on dist-test-slave-q9h9"
server_key: "80aab16ec08c67e92eb751d4118324ce"
server_key_iv: "fe3d76fb620ab4f00bc8194f2c775f7c"
server_key_version: "encryptionkey@0"
I20260806 13:10:24.369966 6971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:24.378839 6971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:24.380369 6971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:24.380470 6971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:24.380622 6971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:24.380865 6971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:24.380895 6971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:24.380913 6971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:24.380929 6971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:24.386097 6971 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:35495
I20260806 13:10:24.386147 7174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:35495 every 8 connection(s)
I20260806 13:10:24.386410 6971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:10:24.389940 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 6971
I20260806 13:10:24.390069 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:10:24.390280 14209 external_mini_cluster.cc:1428] Setting key aa809b44eaa64dc3049d7bfe3ba90ee4
Aug 06 13:10:24 dist-test-slave-q9h9 krb5kdc[6614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021824, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:24.394905 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.386970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:51385 (local address 127.13.224.126:42001)
0806 13:10:24.387011 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:24.387013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:24.387214 (+ 201us) server_negotiation.cc:407] Connection header received
0806 13:10:24.387606 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:24.387607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:24.387616 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:24.387673 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:24.388432 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.388905 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.389779 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.390072 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.390142 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.392180 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:24.392182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:24.392182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:24.392190 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:24.393605 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:24.393962 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:24.393963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:24.393964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:24.394051 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:24.394318 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:24.394319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:24.394319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:24.394454 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:24.394523 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:24.394718 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 13:10:24.394818 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:10:24.394951 7178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.386944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:42001 (local address 127.13.224.66:51385)
0806 13:10:24.387105 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:24.387143 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:24.387537 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:24.387711 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:24.387714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:24.388197 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:24.388372 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.388377 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.388961 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.388961 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.389719 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.389721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.390154 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.390155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.390274 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.390699 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:24.390707 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:24.392090 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:24.393683 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:24.393690 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:24.393692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:24.393890 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:24.394122 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:24.394123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:24.394124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:24.394250 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:24.394512 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:24.394514 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:24.394612 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 13:10:24.394686 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 13:10:24.394797 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260806 13:10:24.395468 7176 heartbeater.cc:347] Connected to a master server at 127.13.224.126:42001
I20260806 13:10:24.395553 7176 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:24.395714 7176 heartbeater.cc:511] Master 127.13.224.126:42001 requested a full tablet report, sending...
I20260806 13:10:24.396067 6690 ts_manager.cc:194] Registered new tserver with Master: 8fdfc54037af48619469828ad8a2454e (127.13.224.66:35495)
I20260806 13:10:24.396544 6690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:51385
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:10:24.413125 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:42001
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:46045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:24.491335 7181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:24.491521 7181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:24.491551 7181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:24.492652 7181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:24.492691 7181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:24.492724 7181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:10:24.494132 7181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:42001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:24.494359 7181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:24.494565 7181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:24.499141 7189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:24.499153 7188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:24.499311 7181 server_base.cc:1034] running on GCE node
W20260806 13:10:24.499236 7192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:24.499526 7181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:24.499743 7181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:24.500877 7181 hybrid_clock.cc:648] HybridClock initialized: now 1786021824500863 us; error 28 us; skew 500 ppm
Aug 06 13:10:24 dist-test-slave-q9h9 krb5kdc[6614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021824, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:24.503149 7181 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:10:24.503553 7181 webserver.cc:460] Webserver started at http://127.13.224.67:43011/ using document root <none> and password file <none>
I20260806 13:10:24.503750 7181 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:24.503796 7181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:24.503916 7181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:24.504827 7181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "a16ae6d6b8ab47fbad6ee12dfe612156"
format_stamp: "Formatted at 2026-08-06 13:10:24 on dist-test-slave-q9h9"
server_key: "19ba15a0b7c3fba36e7d4ea92f707a0e"
server_key_iv: "f5c8ebc45366400f91b2ca9984f1e5d8"
server_key_version: "encryptionkey@0"
I20260806 13:10:24.505131 7181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "a16ae6d6b8ab47fbad6ee12dfe612156"
format_stamp: "Formatted at 2026-08-06 13:10:24 on dist-test-slave-q9h9"
server_key: "19ba15a0b7c3fba36e7d4ea92f707a0e"
server_key_iv: "f5c8ebc45366400f91b2ca9984f1e5d8"
server_key_version: "encryptionkey@0"
I20260806 13:10:24.506317 7181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:10:24.506950 7203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:24.507091 7181 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:24.507171 7181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "a16ae6d6b8ab47fbad6ee12dfe612156"
format_stamp: "Formatted at 2026-08-06 13:10:24 on dist-test-slave-q9h9"
server_key: "19ba15a0b7c3fba36e7d4ea92f707a0e"
server_key_iv: "f5c8ebc45366400f91b2ca9984f1e5d8"
server_key_version: "encryptionkey@0"
I20260806 13:10:24.507228 7181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:24.537267 7181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:24.538970 7181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:24.539089 7181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:24.539278 7181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:24.539532 7181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:24.539564 7181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:24.539589 7181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:24.539613 7181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:24.544914 7181 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:38511
I20260806 13:10:24.544968 7354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:38511 every 8 connection(s)
I20260806 13:10:24.545217 7181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:10:24.547335 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 7181
I20260806 13:10:24.547426 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:10:24.547600 14209 external_mini_cluster.cc:1428] Setting key 33903f8a9de9d18944576483055a5024
Aug 06 13:10:24 dist-test-slave-q9h9 krb5kdc[6614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021824, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:24.553324 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.545815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:56669 (local address 127.13.224.126:42001)
0806 13:10:24.545860 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:24.545862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:24.546053 (+ 191us) server_negotiation.cc:407] Connection header received
0806 13:10:24.546518 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:24.546519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:24.546532 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:24.546584 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:24.547472 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.548108 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.549137 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.549328 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.549374 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.551177 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:24.551179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:24.551179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:24.551186 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:24.552352 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:24.552668 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:24.552669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:24.552669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:24.552714 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:24.552880 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:24.552880 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:24.552880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:24.552970 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:24.553013 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:24.553201 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 13:10:24.553264 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:10:24.553467 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.545782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:42001 (local address 127.13.224.67:56669)
0806 13:10:24.545942 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:24.545972 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:24.546418 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:24.546646 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:24.546650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:24.547200 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:24.547370 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.547374 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.548239 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.548239 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.549060 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.549062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.549384 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.549385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.549478 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.549835 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:24.549842 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:24.551102 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:24.552419 (+ 1317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:24.552425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:24.552426 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:24.552617 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:24.552756 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:24.552757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:24.552757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:24.552839 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:24.553009 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:24.553012 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:24.553111 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 13:10:24.553188 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 13:10:24.553292 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260806 13:10:24.553865 7355 heartbeater.cc:347] Connected to a master server at 127.13.224.126:42001
I20260806 13:10:24.553941 7355 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:24.554102 7355 heartbeater.cc:511] Master 127.13.224.126:42001 requested a full tablet report, sending...
I20260806 13:10:24.554431 6690 ts_manager.cc:194] Registered new tserver with Master: a16ae6d6b8ab47fbad6ee12dfe612156 (127.13.224.67:38511)
I20260806 13:10:24.554759 6690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:56669
I20260806 13:10:24.559098 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:10:24.565121 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.560323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55610 (local address 127.13.224.126:42001)
0806 13:10:24.560369 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:24.560370 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:24.560468 (+ 98us) server_negotiation.cc:407] Connection header received
0806 13:10:24.560508 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:24.560509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:24.560516 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:24.560557 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:24.561046 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.561469 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.562164 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.562332 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.562379 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.563293 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:24.563294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:24.563294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:24.563299 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:24.564332 (+ 1033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:24.564610 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:24.564610 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:24.564610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:24.564649 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:24.564776 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:24.564776 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:24.564776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:24.564884 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:24.564950 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:24.565019 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 13:10:24.565070 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 13:10:24.566350 6690 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:10:24.567771 6690 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:10:24.573442 7381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.569705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:38511 (local address 127.0.0.1:39832)
0806 13:10:24.570043 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:24.570052 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:24.570100 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:24.570305 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:24.570306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:24.570308 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:24.570356 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.570358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.571524 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.571525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.572368 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.572371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.573163 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.573164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.573246 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.573260 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 13:10:24.573320 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 13:10:24.573369 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":77,"threads_started":1}
I20260806 13:10:24.573606 7383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.569771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39832 (local address 127.13.224.67:38511)
0806 13:10:24.569903 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:24.569908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:24.570098 (+ 190us) server_negotiation.cc:407] Connection header received
0806 13:10:24.570155 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:24.570157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:24.570170 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:24.570238 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:24.570429 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.571434 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.572455 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.573076 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.573144 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.573193 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:24.573501 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 13:10:24.573538 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260806 13:10:24.573616 7373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.569637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.13.224.66:35495)
0806 13:10:24.570088 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:24.570092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:24.570101 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:10:24.570138 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:24.570140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:24.570155 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:24.570234 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:24.570314 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.571389 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.572699 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.573163 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.573227 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.573283 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:24.573435 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 13:10:24.573485 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":93,"threads_started":1}
I20260806 13:10:24.573860 7368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.569503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:41329 (local address 127.0.0.1:33210)
0806 13:10:24.569821 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:24.569858 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:24.569921 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:24.570153 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:24.570155 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:24.570157 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:24.570214 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.570216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.571338 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.571339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.572568 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.572571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.573612 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.573613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.573674 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.573678 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:10:24.573708 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 13:10:24.573756 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":63,"threads_started":1}
I20260806 13:10:24.573750 7370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.569518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:35495 (local address 127.0.0.1:55572)
0806 13:10:24.569822 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:24.569860 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:24.569919 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:24.570229 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:24.570230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:24.570232 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:24.570261 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.570262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.571500 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.571501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.572613 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.572615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.573256 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.573257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.573322 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.573326 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:10:24.573384 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 13:10:24.573425 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":49,"threads_started":1}
I20260806 13:10:24.574030 7372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.569620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.13.224.65:41329)
0806 13:10:24.569945 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:24.569949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:24.569970 (+ 21us) server_negotiation.cc:407] Connection header received
0806 13:10:24.570010 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:24.570012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:24.570050 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:24.570288 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:24.570384 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.571250 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.572653 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.573494 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.573801 (+ 307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.573868 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:24.573892 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 13:10:24.573931 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":103,"threads_started":1}
I20260806 13:10:24.574613 7072 tablet_service.cc:1467] Processing CreateTablet for tablet 9dea9a5523b34eef8e6bd297044bc842 (DEFAULT_TABLE table=test-table [id=5e9fd3cc82124a0cb0434742ac6483bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:24.574666 7270 tablet_service.cc:1467] Processing CreateTablet for tablet 9dea9a5523b34eef8e6bd297044bc842 (DEFAULT_TABLE table=test-table [id=5e9fd3cc82124a0cb0434742ac6483bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:24.574764 6869 tablet_service.cc:1467] Processing CreateTablet for tablet 9dea9a5523b34eef8e6bd297044bc842 (DEFAULT_TABLE table=test-table [id=5e9fd3cc82124a0cb0434742ac6483bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:24.574972 7072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dea9a5523b34eef8e6bd297044bc842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:24.574999 7270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dea9a5523b34eef8e6bd297044bc842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:24.575034 6869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dea9a5523b34eef8e6bd297044bc842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:24.577569 7384 tablet_bootstrap.cc:492] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e: Bootstrap starting.
I20260806 13:10:24.577970 7386 tablet_bootstrap.cc:492] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156: Bootstrap starting.
I20260806 13:10:24.578218 7384 tablet_bootstrap.cc:654] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:24.578608 7386 tablet_bootstrap.cc:654] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:24.578812 7384 log.cc:824] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:24.579139 7386 log.cc:824] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:24.579169 7385 tablet_bootstrap.cc:492] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92: Bootstrap starting.
I20260806 13:10:24.579742 7385 tablet_bootstrap.cc:654] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:24.579843 7386 tablet_bootstrap.cc:492] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156: No bootstrap required, opened a new log
I20260806 13:10:24.579916 7386 ts_tablet_manager.cc:1397] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:10:24.580303 7385 log.cc:824] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:24.580865 7384 tablet_bootstrap.cc:492] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e: No bootstrap required, opened a new log
I20260806 13:10:24.580936 7384 ts_tablet_manager.cc:1397] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 13:10:24.581107 7385 tablet_bootstrap.cc:492] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92: No bootstrap required, opened a new log
I20260806 13:10:24.581169 7385 ts_tablet_manager.cc:1397] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:10:24.581741 7386 raft_consensus.cc:348] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3647dacda7fe4c0fadc6a02472ca3b92" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 41329 } } peers { permanent_uuid: "8fdfc54037af48619469828ad8a2454e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 35495 } } peers { permanent_uuid: "a16ae6d6b8ab47fbad6ee12dfe612156" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 38511 } }
I20260806 13:10:24.581920 7386 raft_consensus.cc:374] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:24.581955 7386 raft_consensus.cc:729] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16ae6d6b8ab47fbad6ee12dfe612156, State: Initialized, Role: FOLLOWER
I20260806 13:10:24.582090 7386 consensus_queue.cc:260] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3647dacda7fe4c0fadc6a02472ca3b92" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 41329 } } peers { permanent_uuid: "8fdfc54037af48619469828ad8a2454e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 35495 } } peers { permanent_uuid: "a16ae6d6b8ab47fbad6ee12dfe612156" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 38511 } }
I20260806 13:10:24.582381 7386 ts_tablet_manager.cc:1428] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 13:10:24.582501 7355 heartbeater.cc:503] Master 127.13.224.126:42001 was elected leader, sending a full tablet report...
I20260806 13:10:24.582798 7384 raft_consensus.cc:348] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3647dacda7fe4c0fadc6a02472ca3b92" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 41329 } } peers { permanent_uuid: "8fdfc54037af48619469828ad8a2454e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 35495 } } peers { permanent_uuid: "a16ae6d6b8ab47fbad6ee12dfe612156" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 38511 } }
I20260806 13:10:24.582952 7384 raft_consensus.cc:374] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:24.582994 7384 raft_consensus.cc:729] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fdfc54037af48619469828ad8a2454e, State: Initialized, Role: FOLLOWER
I20260806 13:10:24.583094 7384 consensus_queue.cc:260] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3647dacda7fe4c0fadc6a02472ca3b92" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 41329 } } peers { permanent_uuid: "8fdfc54037af48619469828ad8a2454e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 35495 } } peers { permanent_uuid: "a16ae6d6b8ab47fbad6ee12dfe612156" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 38511 } }
I20260806 13:10:24.583019 7385 raft_consensus.cc:348] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3647dacda7fe4c0fadc6a02472ca3b92" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 41329 } } peers { permanent_uuid: "8fdfc54037af48619469828ad8a2454e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 35495 } } peers { permanent_uuid: "a16ae6d6b8ab47fbad6ee12dfe612156" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 38511 } }
I20260806 13:10:24.583173 7385 raft_consensus.cc:374] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:24.583210 7385 raft_consensus.cc:729] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3647dacda7fe4c0fadc6a02472ca3b92, State: Initialized, Role: FOLLOWER
I20260806 13:10:24.583314 7385 consensus_queue.cc:260] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3647dacda7fe4c0fadc6a02472ca3b92" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 41329 } } peers { permanent_uuid: "8fdfc54037af48619469828ad8a2454e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 35495 } } peers { permanent_uuid: "a16ae6d6b8ab47fbad6ee12dfe612156" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 38511 } }
I20260806 13:10:24.583354 7384 ts_tablet_manager.cc:1428] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 13:10:24.583483 7176 heartbeater.cc:503] Master 127.13.224.126:42001 was elected leader, sending a full tablet report...
I20260806 13:10:24.583557 7385 ts_tablet_manager.cc:1428] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:10:24.583660 6962 heartbeater.cc:503] Master 127.13.224.126:42001 was elected leader, sending a full tablet report...
W20260806 13:10:24.636857 7177 tablet.cc:2369] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 13:10:24.748885 6964 tablet.cc:2369] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:10:24.784739 7394 raft_consensus.cc:482] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:10:24.784833 7394 raft_consensus.cc:504] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3647dacda7fe4c0fadc6a02472ca3b92" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 41329 } } peers { permanent_uuid: "8fdfc54037af48619469828ad8a2454e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 35495 } } peers { permanent_uuid: "a16ae6d6b8ab47fbad6ee12dfe612156" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 38511 } }
I20260806 13:10:24.785142 7394 leader_election.cc:288] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3647dacda7fe4c0fadc6a02472ca3b92 (127.13.224.65:41329), a16ae6d6b8ab47fbad6ee12dfe612156 (127.13.224.67:38511)
I20260806 13:10:24.787803 7178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.785296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:41329 (local address 127.13.224.66:58395)
0806 13:10:24.785357 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:24.785368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:24.785419 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:24.785557 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:24.785558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:24.785560 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:24.785630 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.785632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.786318 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.786319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.787067 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.787069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.787565 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.787566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.787642 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.787647 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:10:24.787706 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 13:10:24.787740 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 13:10:24.787822 7372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.785368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:58395 (local address 127.13.224.65:41329)
0806 13:10:24.785406 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:24.785408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:24.785417 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:10:24.785469 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:24.785470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:24.785483 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:24.785535 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:24.785708 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.786253 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.787145 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.787499 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.787549 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.787555 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:24.787709 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 13:10:24.787757 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:10:24.787900 7399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.785374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:38511 (local address 127.13.224.66:45431)
0806 13:10:24.785529 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:24.785537 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:24.785580 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:24.785699 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:24.785700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:24.785701 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:24.785772 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.785773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.786410 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.786411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.787120 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:24.787122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.787667 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.787668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:24.787737 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.787742 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:10:24.787796 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 13:10:24.787831 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260806 13:10:24.787904 7383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.785469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:45431 (local address 127.13.224.67:38511)
0806 13:10:24.785525 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:24.785527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:24.785572 (+ 45us) server_negotiation.cc:407] Connection header received
0806 13:10:24.785623 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:24.785624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:24.785632 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:24.785691 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:24.785834 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.786338 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.787211 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.787596 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.787660 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.787668 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:24.787807 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 13:10:24.787848 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 13:10:24.788260 7299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dea9a5523b34eef8e6bd297044bc842" candidate_uuid: "8fdfc54037af48619469828ad8a2454e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16ae6d6b8ab47fbad6ee12dfe612156" is_pre_election: true
I20260806 13:10:24.788275 6897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dea9a5523b34eef8e6bd297044bc842" candidate_uuid: "8fdfc54037af48619469828ad8a2454e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3647dacda7fe4c0fadc6a02472ca3b92" is_pre_election: true
I20260806 13:10:24.788389 7299 raft_consensus.cc:2454] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fdfc54037af48619469828ad8a2454e in term 0.
I20260806 13:10:24.788396 6897 raft_consensus.cc:2454] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fdfc54037af48619469828ad8a2454e in term 0.
I20260806 13:10:24.788596 6998 leader_election.cc:302] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fdfc54037af48619469828ad8a2454e, a16ae6d6b8ab47fbad6ee12dfe612156; no voters:
I20260806 13:10:24.788729 7394 raft_consensus.cc:2781] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:10:24.788792 7394 raft_consensus.cc:482] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:10:24.788807 7394 raft_consensus.cc:3037] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:24.789431 7394 raft_consensus.cc:504] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3647dacda7fe4c0fadc6a02472ca3b92" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 41329 } } peers { permanent_uuid: "8fdfc54037af48619469828ad8a2454e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 35495 } } peers { permanent_uuid: "a16ae6d6b8ab47fbad6ee12dfe612156" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 38511 } }
I20260806 13:10:24.789531 7394 leader_election.cc:288] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [CANDIDATE]: Term 1 election: Requested vote from peers 3647dacda7fe4c0fadc6a02472ca3b92 (127.13.224.65:41329), a16ae6d6b8ab47fbad6ee12dfe612156 (127.13.224.67:38511)
I20260806 13:10:24.789698 6897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dea9a5523b34eef8e6bd297044bc842" candidate_uuid: "8fdfc54037af48619469828ad8a2454e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3647dacda7fe4c0fadc6a02472ca3b92"
I20260806 13:10:24.789723 7299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dea9a5523b34eef8e6bd297044bc842" candidate_uuid: "8fdfc54037af48619469828ad8a2454e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a16ae6d6b8ab47fbad6ee12dfe612156"
I20260806 13:10:24.789770 6897 raft_consensus.cc:3037] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:24.789784 7299 raft_consensus.cc:3037] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:24.790508 6897 raft_consensus.cc:2454] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fdfc54037af48619469828ad8a2454e in term 1.
I20260806 13:10:24.790515 7299 raft_consensus.cc:2454] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fdfc54037af48619469828ad8a2454e in term 1.
I20260806 13:10:24.790664 6998 leader_election.cc:302] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3647dacda7fe4c0fadc6a02472ca3b92, 8fdfc54037af48619469828ad8a2454e; no voters:
I20260806 13:10:24.790752 7394 raft_consensus.cc:2781] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:24.790868 7394 raft_consensus.cc:686] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [term 1 LEADER]: Becoming Leader. State: Replica: 8fdfc54037af48619469828ad8a2454e, State: Running, Role: LEADER
I20260806 13:10:24.790946 7394 consensus_queue.cc:237] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3647dacda7fe4c0fadc6a02472ca3b92" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 41329 } } peers { permanent_uuid: "8fdfc54037af48619469828ad8a2454e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 35495 } } peers { permanent_uuid: "a16ae6d6b8ab47fbad6ee12dfe612156" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 38511 } }
I20260806 13:10:24.791702 6688 catalog_manager.cc:5495] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fdfc54037af48619469828ad8a2454e (127.13.224.66). New cstate: current_term: 1 leader_uuid: "8fdfc54037af48619469828ad8a2454e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3647dacda7fe4c0fadc6a02472ca3b92" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 41329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fdfc54037af48619469828ad8a2454e" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 35495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a16ae6d6b8ab47fbad6ee12dfe612156" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 38511 } health_report { overall_health: UNKNOWN } } }
W20260806 13:10:24.795674 7356 tablet.cc:2369] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:10:24.851711 7373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.849522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55588 (local address 127.13.224.66:35495)
0806 13:10:24.849574 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:24.849576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:24.849583 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:10:24.849635 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:24.849635 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:24.849652 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:24.849709 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:24.849834 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.850305 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.851031 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:24.851213 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:24.851266 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:24.851412 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:24.851502 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:24.851570 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:24.851608 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 13:10:24.851625 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 13:10:24.853303 6897 raft_consensus.cc:1261] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92 [term 1 FOLLOWER]: Refusing update from remote peer 8fdfc54037af48619469828ad8a2454e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:24.853303 7299 raft_consensus.cc:1261] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156 [term 1 FOLLOWER]: Refusing update from remote peer 8fdfc54037af48619469828ad8a2454e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:24.853653 7401 consensus_queue.cc:1035] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3647dacda7fe4c0fadc6a02472ca3b92" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 41329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:24.853780 7394 consensus_queue.cc:1035] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a16ae6d6b8ab47fbad6ee12dfe612156" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 38511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:24.856980 7406 mvcc.cc:205] Tried to move back new op lower bound from 7315545394596737024 to 7315545394344218624. Current Snapshot: MvccSnapshot[applied={T|T < 7315545394596737024}]
I20260806 13:10:24.857888 6688 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:10:24.857961 6688 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:10:24.858644 6688 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3d3efb1b9be46a1a96d28699cbf3fb4: Sending DeleteTablet for 3 replicas of tablet 9dea9a5523b34eef8e6bd297044bc842
I20260806 13:10:24.858974 6869 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dea9a5523b34eef8e6bd297044bc842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:24 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:33210
I20260806 13:10:24.859002 7072 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dea9a5523b34eef8e6bd297044bc842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:24 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:55572
I20260806 13:10:24.859193 7270 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dea9a5523b34eef8e6bd297044bc842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:24 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:39832
I20260806 13:10:24.859269 7418 tablet_replica.cc:330] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92: stopping tablet replica
I20260806 13:10:24.859359 7418 raft_consensus.cc:2229] T 9dea9a5523b34eef8e6bd297044bc842 P 3647dacda7fe4c0fadc6a02472ca3b92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:10:24.859424 7421 tablet_replica.cc:330] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156: stopping tablet replica
I20260806 13:10:24.859525 7420 tablet_replica.cc:330] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e: stopping tablet replica
I20260806 13:10:24.859555 7421 raft_consensus.cc:2229] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:10:24.859643 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 6778
I20260806 13:10:24.859686 7421 raft_consensus.cc:2258] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:24.889645 7420 raft_consensus.cc:2229] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:10:24.890271 7421 ts_tablet_manager.cc:1905] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:10:24.890473 7420 raft_consensus.cc:2258] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:24.890946 7420 ts_tablet_manager.cc:1905] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:10:24.891924 7421 ts_tablet_manager.cc:1918] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:10:24.892048 7421 log.cc:1196] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/9dea9a5523b34eef8e6bd297044bc842
I20260806 13:10:24.892493 7421 ts_tablet_manager.cc:1939] T 9dea9a5523b34eef8e6bd297044bc842 P a16ae6d6b8ab47fbad6ee12dfe612156: Deleting consensus metadata
I20260806 13:10:24.892521 7420 ts_tablet_manager.cc:1918] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:10:24.892602 7420 log.cc:1196] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/9dea9a5523b34eef8e6bd297044bc842
I20260806 13:10:24.892856 7420 ts_tablet_manager.cc:1939] T 9dea9a5523b34eef8e6bd297044bc842 P 8fdfc54037af48619469828ad8a2454e: Deleting consensus metadata
I20260806 13:10:24.893064 6666 catalog_manager.cc:4841] TS a16ae6d6b8ab47fbad6ee12dfe612156 (127.13.224.67:38511): tablet 9dea9a5523b34eef8e6bd297044bc842 (table test-table [id=5e9fd3cc82124a0cb0434742ac6483bb]) successfully deleted
I20260806 13:10:24.893260 6663 catalog_manager.cc:4841] TS 8fdfc54037af48619469828ad8a2454e (127.13.224.66:35495): tablet 9dea9a5523b34eef8e6bd297044bc842 (table test-table [id=5e9fd3cc82124a0cb0434742ac6483bb]) successfully deleted
W20260806 13:10:24.894851 6666 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:41329 (error 108)
I20260806 13:10:24.895116 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 6971
I20260806 13:10:24.895262 7381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:24.895145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:41329 (local address 127.0.0.1:33224)
0806 13:10:24.895192 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:24.895225 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 13:10:24.906950 6666 catalog_manager.cc:4584] TS 3647dacda7fe4c0fadc6a02472ca3b92 (127.13.224.65:41329): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9dea9a5523b34eef8e6bd297044bc842: Network error: Client connection negotiation failed: client connection to 127.13.224.65:41329: connect: Connection refused (error 111)
I20260806 13:10:24.914515 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 7181
I20260806 13:10:24.922153 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 6635
2026-08-06T13:10:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2197 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T13:10:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:24Z Disabled control of system clock
I20260806 13:10:24.953320 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:37251
--webserver_interface=127.13.224.126
--webserver_port=0
--builtin_ntp_servers=127.13.224.84:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:37251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:10:25.032150 7427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:25.032339 7427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:25.032366 7427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:25.033445 7427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:25.033484 7427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:25.033502 7427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:25.033517 7427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:25.033533 7427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:25.034921 7427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:37251
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:37251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:25.035145 7427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:25.035393 7427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:25.039793 7444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:25.039887 7427 server_base.cc:1034] running on GCE node
W20260806 13:10:25.039793 7441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:25.039994 7442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:25.040283 7427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:25.040504 7427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:25.041644 7427 hybrid_clock.cc:648] HybridClock initialized: now 1786021825041632 us; error 28 us; skew 500 ppm
I20260806 13:10:25.042724 7427 webserver.cc:460] Webserver started at http://127.13.224.126:36489/ using document root <none> and password file <none>
I20260806 13:10:25.042941 7427 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:25.042985 7427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:25.043103 7427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:25.044075 7427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "3451015d2e364c05aa4601e200584908"
format_stamp: "Formatted at 2026-08-06 13:10:25 on dist-test-slave-q9h9"
server_key: "0347a9118b0deaf2f4c033b86b4f058b"
server_key_iv: "305ff7164a7bd85c82c5c3542f1408db"
server_key_version: "encryptionkey@0"
I20260806 13:10:25.044384 7427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "3451015d2e364c05aa4601e200584908"
format_stamp: "Formatted at 2026-08-06 13:10:25 on dist-test-slave-q9h9"
server_key: "0347a9118b0deaf2f4c033b86b4f058b"
server_key_iv: "305ff7164a7bd85c82c5c3542f1408db"
server_key_version: "encryptionkey@0"
I20260806 13:10:25.045499 7427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:10:25.046137 7450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:25.046290 7427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:25.046372 7427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "3451015d2e364c05aa4601e200584908"
format_stamp: "Formatted at 2026-08-06 13:10:25 on dist-test-slave-q9h9"
server_key: "0347a9118b0deaf2f4c033b86b4f058b"
server_key_iv: "305ff7164a7bd85c82c5c3542f1408db"
server_key_version: "encryptionkey@0"
I20260806 13:10:25.046423 7427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:25.054442 7427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:25.054651 7427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:25.054760 7427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:25.058218 7427 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:37251
I20260806 13:10:25.058260 7532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:37251 every 8 connection(s)
I20260806 13:10:25.058521 7427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:25.059379 7534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:25.061347 7534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908: Bootstrap starting.
I20260806 13:10:25.061872 7534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:25.062179 7534 log.cc:824] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:25.062769 7534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908: No bootstrap required, opened a new log
I20260806 13:10:25.064214 7534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3451015d2e364c05aa4601e200584908" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37251 } }
I20260806 13:10:25.064334 7534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:25.064366 7534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3451015d2e364c05aa4601e200584908, State: Initialized, Role: FOLLOWER
I20260806 13:10:25.064496 7534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3451015d2e364c05aa4601e200584908" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37251 } }
I20260806 13:10:25.064553 7534 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:25.064574 7534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:25.064617 7534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:25.065140 7534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3451015d2e364c05aa4601e200584908" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37251 } }
I20260806 13:10:25.065229 7534 leader_election.cc:302] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3451015d2e364c05aa4601e200584908; no voters:
I20260806 13:10:25.065379 7534 leader_election.cc:288] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:25.065419 7539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:25.065558 7539 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [term 1 LEADER]: Becoming Leader. State: Replica: 3451015d2e364c05aa4601e200584908, State: Running, Role: LEADER
I20260806 13:10:25.065610 7534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:25.065655 7539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3451015d2e364c05aa4601e200584908" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37251 } }
I20260806 13:10:25.065987 7541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3451015d2e364c05aa4601e200584908. Latest consensus state: current_term: 1 leader_uuid: "3451015d2e364c05aa4601e200584908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3451015d2e364c05aa4601e200584908" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37251 } } }
I20260806 13:10:25.066092 7541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:25.066196 7540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3451015d2e364c05aa4601e200584908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3451015d2e364c05aa4601e200584908" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37251 } } }
I20260806 13:10:25.066567 7549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:25.066628 7540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:25.067112 7549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:25.068064 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 7427
I20260806 13:10:25.068168 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:25.068372 14209 external_mini_cluster.cc:1428] Setting key 296d833ba127c0d8deea199241652fa1
I20260806 13:10:25.069070 7549 catalog_manager.cc:1347] Generated new cluster ID: 8d480a0dd491454ebd1714f9bf396bd5
I20260806 13:10:25.069113 7549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:25.074138 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.068770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55784 (local address 127.13.224.126:37251)
0806 13:10:25.069150 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:25.069158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:25.069187 (+ 29us) server_negotiation.cc:407] Connection header received
0806 13:10:25.069301 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:25.069307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:25.069380 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:25.069519 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:25.069729 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.071094 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.072108 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.072720 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.072841 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.073418 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:25.073422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:25.073423 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:25.073633 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:25.073676 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:25.073692 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:25.073766 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:25.073825 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 13:10:25.073910 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":71,"threads_started":1}
I20260806 13:10:25.075351 7549 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:25.075801 7549 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:25.087718 7549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908: Generated new TSK 0
I20260806 13:10:25.087859 7549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:25.089053 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37251
--builtin_ntp_servers=127.13.224.84:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:10:25.172058 7567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:25.172250 7567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:25.172271 7567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:25.173368 7567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:25.173405 7567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:25.173429 7567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:10:25.174796 7567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:25.175032 7567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:25.175310 7567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:25.180686 7578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:25.180730 7580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:25.180725 7576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:25.180902 7567 server_base.cc:1034] running on GCE node
I20260806 13:10:25.181107 7567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:25.181366 7567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:25.182531 7567 hybrid_clock.cc:648] HybridClock initialized: now 1786021825182519 us; error 27 us; skew 500 ppm
I20260806 13:10:25.183914 7567 webserver.cc:460] Webserver started at http://127.13.224.65:37569/ using document root <none> and password file <none>
I20260806 13:10:25.184206 7567 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:25.184278 7567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:25.184427 7567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:25.185756 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "47e633bcc1bd4d89affc004b36b7dad7"
format_stamp: "Formatted at 2026-08-06 13:10:25 on dist-test-slave-q9h9"
server_key: "f7d49c118d6a30600be0f68bc382ae74"
server_key_iv: "841f6e1014dc96825a6506d4bef3bd7e"
server_key_version: "encryptionkey@0"
I20260806 13:10:25.186180 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "47e633bcc1bd4d89affc004b36b7dad7"
format_stamp: "Formatted at 2026-08-06 13:10:25 on dist-test-slave-q9h9"
server_key: "f7d49c118d6a30600be0f68bc382ae74"
server_key_iv: "841f6e1014dc96825a6506d4bef3bd7e"
server_key_version: "encryptionkey@0"
I20260806 13:10:25.187798 7567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:10:25.188650 7588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:25.188803 7567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:10:25.188901 7567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "47e633bcc1bd4d89affc004b36b7dad7"
format_stamp: "Formatted at 2026-08-06 13:10:25 on dist-test-slave-q9h9"
server_key: "f7d49c118d6a30600be0f68bc382ae74"
server_key_iv: "841f6e1014dc96825a6506d4bef3bd7e"
server_key_version: "encryptionkey@0"
I20260806 13:10:25.188966 7567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:25.197712 7567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:25.197916 7567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:25.198005 7567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:25.198220 7567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:25.198485 7567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:25.198518 7567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:25.198536 7567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:25.198559 7567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:25.203974 7567 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:33873
I20260806 13:10:25.204010 7754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:33873 every 8 connection(s)
I20260806 13:10:25.204329 7567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:10:25.204823 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 7567
I20260806 13:10:25.204906 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:10:25.205117 14209 external_mini_cluster.cc:1428] Setting key ddfeb63ba7401a4a21cadca1e9a8845e
I20260806 13:10:25.206121 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37251
--builtin_ntp_servers=127.13.224.84:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 13:10:25.209458 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.204956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:46787 (local address 127.13.224.126:37251)
0806 13:10:25.205000 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:25.205002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:25.205260 (+ 258us) server_negotiation.cc:407] Connection header received
0806 13:10:25.205780 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:25.205781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:25.205793 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:25.205844 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:25.206643 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.207110 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.208229 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.208511 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.208581 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.209222 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:25.209223 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:25.209224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:25.209228 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:25.209253 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:25.209258 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:25.209308 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:25.209359 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 13:10:25.209384 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:10:25.209653 7758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.204950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37251 (local address 127.13.224.65:46787)
0806 13:10:25.205130 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:25.205172 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:25.205701 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:25.205910 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:25.205914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:25.205925 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:25.206560 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.206567 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.207189 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.207190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.208151 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.208154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.208581 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.208581 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.208676 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.209103 (+ 427us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:10:25.209104 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:25.209129 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:10:25.209149 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:25.209312 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:25.209314 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:25.209317 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:10:25.209358 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 13:10:25.209425 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260806 13:10:25.210083 7755 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37251
I20260806 13:10:25.210170 7755 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:25.210341 7755 heartbeater.cc:511] Master 127.13.224.126:37251 requested a full tablet report, sending...
I20260806 13:10:25.210852 7479 ts_manager.cc:194] Registered new tserver with Master: 47e633bcc1bd4d89affc004b36b7dad7 (127.13.224.65:33873)
I20260806 13:10:25.211681 7479 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.224.65:46787
W20260806 13:10:25.302678 7759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:25.302860 7759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:25.302879 7759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:25.303970 7759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:25.304008 7759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:25.304030 7759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:10:25.305365 7759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:25.305583 7759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:25.305850 7759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:25.310336 7768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:25.310333 7771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:25.310353 7767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:25.310987 7759 server_base.cc:1034] running on GCE node
I20260806 13:10:25.311151 7759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:25.311367 7759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:25.312496 7759 hybrid_clock.cc:648] HybridClock initialized: now 1786021825312483 us; error 28 us; skew 500 ppm
I20260806 13:10:25.313591 7759 webserver.cc:460] Webserver started at http://127.13.224.66:36503/ using document root <none> and password file <none>
I20260806 13:10:25.313809 7759 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:25.313853 7759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:25.313973 7759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:25.314949 7759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "a236ca713225496e87549e9b6f2e8251"
format_stamp: "Formatted at 2026-08-06 13:10:25 on dist-test-slave-q9h9"
server_key: "e5353754b6b8c8a900e2ddcf5dbcb95a"
server_key_iv: "395ca0c6a931b6fee6b8684be67eaabb"
server_key_version: "encryptionkey@0"
I20260806 13:10:25.315266 7759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "a236ca713225496e87549e9b6f2e8251"
format_stamp: "Formatted at 2026-08-06 13:10:25 on dist-test-slave-q9h9"
server_key: "e5353754b6b8c8a900e2ddcf5dbcb95a"
server_key_iv: "395ca0c6a931b6fee6b8684be67eaabb"
server_key_version: "encryptionkey@0"
I20260806 13:10:25.316417 7759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:25.317052 7780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:25.317179 7759 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:25.317257 7759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "a236ca713225496e87549e9b6f2e8251"
format_stamp: "Formatted at 2026-08-06 13:10:25 on dist-test-slave-q9h9"
server_key: "e5353754b6b8c8a900e2ddcf5dbcb95a"
server_key_iv: "395ca0c6a931b6fee6b8684be67eaabb"
server_key_version: "encryptionkey@0"
I20260806 13:10:25.317322 7759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:25.336508 7759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:25.336737 7759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:25.336863 7759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:25.337047 7759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:25.337320 7759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:25.337354 7759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:25.337379 7759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:25.337397 7759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:25.342638 7759 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:40727
I20260806 13:10:25.342672 7958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:40727 every 8 connection(s)
I20260806 13:10:25.342936 7759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:10:25.347211 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.343527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:41705 (local address 127.13.224.126:37251)
0806 13:10:25.343583 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:25.343585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:25.343787 (+ 202us) server_negotiation.cc:407] Connection header received
0806 13:10:25.344209 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:25.344210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:25.344221 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:25.344265 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:25.344827 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.345323 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.346181 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.346358 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.346400 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.347037 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:25.347037 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:25.347038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:25.347041 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:25.347050 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:25.347052 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:25.347079 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:25.347133 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 13:10:25.347150 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 13:10:25.347375 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.343516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37251 (local address 127.13.224.66:41705)
0806 13:10:25.343678 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:25.343709 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:25.344126 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:25.344317 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:25.344320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:25.344328 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:25.344772 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.344776 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.345380 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.345381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.346126 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.346128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.346410 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.346411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.346509 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.346934 (+ 425us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:10:25.346935 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:25.346960 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:10:25.346979 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:25.347085 (+ 106us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:25.347087 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:25.347089 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:10:25.347130 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 13:10:25.347197 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":84,"threads_started":1}
I20260806 13:10:25.347770 7959 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37251
I20260806 13:10:25.347862 7959 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:25.347996 7959 heartbeater.cc:511] Master 127.13.224.126:37251 requested a full tablet report, sending...
I20260806 13:10:25.348331 7479 ts_manager.cc:194] Registered new tserver with Master: a236ca713225496e87549e9b6f2e8251 (127.13.224.66:40727)
I20260806 13:10:25.348711 7479 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.224.66:41705
I20260806 13:10:25.351759 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 7759
I20260806 13:10:25.351850 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:10:25.352025 14209 external_mini_cluster.cc:1428] Setting key cf1f1d7e9c92e2832ac8f7e577969370
I20260806 13:10:25.352778 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37251
--builtin_ntp_servers=127.13.224.84:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:10:25.430754 7966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:25.430925 7966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:25.430944 7966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:25.432082 7966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:25.432127 7966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:25.432152 7966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:10:25.433467 7966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:25.433673 7966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:25.433928 7966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:25.438529 7975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:25.438537 7980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:25.438647 7966 server_base.cc:1034] running on GCE node
W20260806 13:10:25.438576 7977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:25.438948 7966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:25.439131 7966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:25.440261 7966 hybrid_clock.cc:648] HybridClock initialized: now 1786021825440254 us; error 31 us; skew 500 ppm
I20260806 13:10:25.441352 7966 webserver.cc:460] Webserver started at http://127.13.224.67:37477/ using document root <none> and password file <none>
I20260806 13:10:25.441550 7966 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:25.441587 7966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:25.441675 7966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:25.442621 7966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "00d7f0739070469684bd2bba9de15ff3"
format_stamp: "Formatted at 2026-08-06 13:10:25 on dist-test-slave-q9h9"
server_key: "f90193ea9e1818a4ab90aaa42a49bf6d"
server_key_iv: "bbf926d9bd2e37fccdbc3ae17ed16236"
server_key_version: "encryptionkey@0"
I20260806 13:10:25.442893 7966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "00d7f0739070469684bd2bba9de15ff3"
format_stamp: "Formatted at 2026-08-06 13:10:25 on dist-test-slave-q9h9"
server_key: "f90193ea9e1818a4ab90aaa42a49bf6d"
server_key_iv: "bbf926d9bd2e37fccdbc3ae17ed16236"
server_key_version: "encryptionkey@0"
I20260806 13:10:25.444015 7966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:25.444621 7987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:25.444782 7966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:25.444859 7966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "00d7f0739070469684bd2bba9de15ff3"
format_stamp: "Formatted at 2026-08-06 13:10:25 on dist-test-slave-q9h9"
server_key: "f90193ea9e1818a4ab90aaa42a49bf6d"
server_key_iv: "bbf926d9bd2e37fccdbc3ae17ed16236"
server_key_version: "encryptionkey@0"
I20260806 13:10:25.444911 7966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:25.464238 7966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:25.464454 7966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:25.464543 7966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:25.464697 7966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:25.464932 7966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:25.464960 7966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:25.464984 7966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:25.465000 7966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:25.470255 7966 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:45001
I20260806 13:10:25.470328 8156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:45001 every 8 connection(s)
I20260806 13:10:25.470573 7966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:10:25.474934 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.471178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:40261 (local address 127.13.224.126:37251)
0806 13:10:25.471234 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:25.471236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:25.471443 (+ 207us) server_negotiation.cc:407] Connection header received
0806 13:10:25.471884 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:25.471885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:25.471895 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:25.471940 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:25.472505 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.472978 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.473851 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.474052 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.474093 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.474712 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:25.474714 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:25.474714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:25.474718 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:25.474731 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:25.474734 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:25.474772 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:25.474852 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 13:10:25.474876 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 13:10:25.475057 8161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.471152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37251 (local address 127.13.224.67:40261)
0806 13:10:25.471329 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:25.471366 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:25.471812 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:25.471994 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:25.471997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:25.472005 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:25.472447 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.472451 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.473040 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.473041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.473793 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.473795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.474102 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.474102 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.474207 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.474593 (+ 386us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:10:25.474593 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:25.474617 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:10:25.474644 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:25.474774 (+ 130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:25.474776 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:25.474778 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:10:25.474829 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 13:10:25.474898 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260806 13:10:25.475476 8158 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37251
I20260806 13:10:25.475560 8158 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:25.475720 8158 heartbeater.cc:511] Master 127.13.224.126:37251 requested a full tablet report, sending...
I20260806 13:10:25.476048 7479 ts_manager.cc:194] Registered new tserver with Master: 00d7f0739070469684bd2bba9de15ff3 (127.13.224.67:45001)
I20260806 13:10:25.476388 7479 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.224.67:40261
I20260806 13:10:25.477598 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 7966
I20260806 13:10:25.477681 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:10:25.477829 14209 external_mini_cluster.cc:1428] Setting key d32bb9c0b432328e81ba808e00639547
I20260806 13:10:25.478567 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:10:25.482050 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.479779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55798 (local address 127.13.224.126:37251)
0806 13:10:25.479841 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:25.479843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:25.479914 (+ 71us) server_negotiation.cc:407] Connection header received
0806 13:10:25.479971 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:25.479971 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:25.479981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:25.480015 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:25.480135 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.480569 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.481237 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.481404 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.481442 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.481872 (+ 430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:25.481873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:25.481874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:25.481877 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:25.481888 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:25.481892 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:25.481927 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:25.481955 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 13:10:25.481973 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 13:10:25.483330 7479 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:10:25.484746 7479 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:10:25.490458 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.486982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37734 (local address 127.13.224.67:45001)
0806 13:10:25.487135 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:25.487160 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:25.487168 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:10:25.487196 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:25.487197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:25.487209 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:25.487280 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:25.487502 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.488206 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.489561 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.490104 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.490167 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.490250 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:25.490348 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 13:10:25.490383 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260806 13:10:25.490675 8169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.486648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:40727 (local address 127.0.0.1:60192)
0806 13:10:25.486800 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:25.486833 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:25.486894 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:25.487312 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:25.487313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:25.487315 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:25.487360 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.487362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.488596 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.488597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.489361 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.489364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.490290 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.490292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.490348 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.490352 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:10:25.490379 (+ 27us) client_negotiation.cc:240] Negotiation successful
0806 13:10:25.490417 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":58,"threads_started":1}
I20260806 13:10:25.490724 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.486814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60192 (local address 127.13.224.66:40727)
0806 13:10:25.487142 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:25.487145 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:25.487154 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:10:25.487189 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:25.487191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:25.487203 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:25.487429 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:25.487484 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.488538 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.489447 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.490190 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.490498 (+ 308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.490584 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:25.490608 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 13:10:25.490647 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":83,"threads_started":1}
I20260806 13:10:25.490828 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.486932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56400 (local address 127.13.224.65:33873)
0806 13:10:25.487082 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:25.487087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:25.487099 (+ 12us) server_negotiation.cc:407] Connection header received
0806 13:10:25.487138 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:25.487140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:25.487155 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:25.487237 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:25.487379 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.488452 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.489631 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.490347 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.490407 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.490510 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:25.490694 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 13:10:25.490738 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260806 13:10:25.490746 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.486673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:33873 (local address 127.0.0.1:56400)
0806 13:10:25.487046 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:25.487054 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:25.487087 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:25.487255 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:25.487257 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:25.487260 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:25.487320 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.487322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.488532 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.488533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.489567 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.489568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.490545 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.490546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.490601 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.490605 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:10:25.490657 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 13:10:25.490693 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":88,"threads_started":1}
I20260806 13:10:25.490458 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.486648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:45001 (local address 127.0.0.1:37734)
0806 13:10:25.486913 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:25.486922 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:25.486972 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:25.487333 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:25.487334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:25.487336 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:25.487432 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.487434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.488280 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.488281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.489471 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.489474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.490177 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.490178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.490255 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.490269 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 13:10:25.490330 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 13:10:25.490383 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260806 13:10:25.491773 7858 tablet_service.cc:1467] Processing CreateTablet for tablet 94ee7fb27eb0407baa676657da0845fe (DEFAULT_TABLE table=test-table [id=59c62c1e11a84cb49257b7c79e2f7141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:25.491756 8051 tablet_service.cc:1467] Processing CreateTablet for tablet 94ee7fb27eb0407baa676657da0845fe (DEFAULT_TABLE table=test-table [id=59c62c1e11a84cb49257b7c79e2f7141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:25.491731 7665 tablet_service.cc:1467] Processing CreateTablet for tablet 94ee7fb27eb0407baa676657da0845fe (DEFAULT_TABLE table=test-table [id=59c62c1e11a84cb49257b7c79e2f7141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:25.492093 7665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94ee7fb27eb0407baa676657da0845fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:25.492093 8051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94ee7fb27eb0407baa676657da0845fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:25.492093 7858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94ee7fb27eb0407baa676657da0845fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:25.494793 8181 tablet_bootstrap.cc:492] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251: Bootstrap starting.
I20260806 13:10:25.495282 8183 tablet_bootstrap.cc:492] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7: Bootstrap starting.
I20260806 13:10:25.495425 8181 tablet_bootstrap.cc:654] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:25.495772 8181 log.cc:824] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:25.496006 8183 tablet_bootstrap.cc:654] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:25.496531 8181 tablet_bootstrap.cc:492] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251: No bootstrap required, opened a new log
I20260806 13:10:25.496585 8183 log.cc:824] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:25.496606 8181 ts_tablet_manager.cc:1397] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:10:25.496903 8182 tablet_bootstrap.cc:492] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3: Bootstrap starting.
I20260806 13:10:25.497233 8183 tablet_bootstrap.cc:492] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7: No bootstrap required, opened a new log
I20260806 13:10:25.497296 8183 ts_tablet_manager.cc:1397] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:10:25.497542 8182 tablet_bootstrap.cc:654] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:25.497867 8182 log.cc:824] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:25.498829 8183 raft_consensus.cc:348] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7f0739070469684bd2bba9de15ff3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45001 } } peers { permanent_uuid: "a236ca713225496e87549e9b6f2e8251" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 40727 } } peers { permanent_uuid: "47e633bcc1bd4d89affc004b36b7dad7" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 33873 } }
I20260806 13:10:25.498955 8183 raft_consensus.cc:374] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:25.498986 8183 raft_consensus.cc:729] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47e633bcc1bd4d89affc004b36b7dad7, State: Initialized, Role: FOLLOWER
I20260806 13:10:25.498957 8181 raft_consensus.cc:348] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7f0739070469684bd2bba9de15ff3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45001 } } peers { permanent_uuid: "a236ca713225496e87549e9b6f2e8251" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 40727 } } peers { permanent_uuid: "47e633bcc1bd4d89affc004b36b7dad7" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 33873 } }
I20260806 13:10:25.499094 8183 consensus_queue.cc:260] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7f0739070469684bd2bba9de15ff3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45001 } } peers { permanent_uuid: "a236ca713225496e87549e9b6f2e8251" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 40727 } } peers { permanent_uuid: "47e633bcc1bd4d89affc004b36b7dad7" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 33873 } }
I20260806 13:10:25.499135 8181 raft_consensus.cc:374] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:25.499177 8181 raft_consensus.cc:729] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a236ca713225496e87549e9b6f2e8251, State: Initialized, Role: FOLLOWER
I20260806 13:10:25.499287 8181 consensus_queue.cc:260] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7f0739070469684bd2bba9de15ff3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45001 } } peers { permanent_uuid: "a236ca713225496e87549e9b6f2e8251" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 40727 } } peers { permanent_uuid: "47e633bcc1bd4d89affc004b36b7dad7" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 33873 } }
I20260806 13:10:25.499328 8183 ts_tablet_manager.cc:1428] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:10:25.499352 7755 heartbeater.cc:503] Master 127.13.224.126:37251 was elected leader, sending a full tablet report...
I20260806 13:10:25.499524 8181 ts_tablet_manager.cc:1428] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:10:25.499547 7959 heartbeater.cc:503] Master 127.13.224.126:37251 was elected leader, sending a full tablet report...
I20260806 13:10:25.500064 8182 tablet_bootstrap.cc:492] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3: No bootstrap required, opened a new log
I20260806 13:10:25.500133 8182 ts_tablet_manager.cc:1397] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 13:10:25.502095 8182 raft_consensus.cc:348] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7f0739070469684bd2bba9de15ff3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45001 } } peers { permanent_uuid: "a236ca713225496e87549e9b6f2e8251" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 40727 } } peers { permanent_uuid: "47e633bcc1bd4d89affc004b36b7dad7" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 33873 } }
I20260806 13:10:25.502218 8182 raft_consensus.cc:374] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:25.502238 8182 raft_consensus.cc:729] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d7f0739070469684bd2bba9de15ff3, State: Initialized, Role: FOLLOWER
I20260806 13:10:25.502319 8182 consensus_queue.cc:260] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7f0739070469684bd2bba9de15ff3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45001 } } peers { permanent_uuid: "a236ca713225496e87549e9b6f2e8251" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 40727 } } peers { permanent_uuid: "47e633bcc1bd4d89affc004b36b7dad7" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 33873 } }
I20260806 13:10:25.502508 8182 ts_tablet_manager.cc:1428] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 13:10:25.502579 8158 heartbeater.cc:503] Master 127.13.224.126:37251 was elected leader, sending a full tablet report...
W20260806 13:10:25.593405 7962 tablet.cc:2369] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:10:25.621590 8187 raft_consensus.cc:482] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:10:25.621701 8187 raft_consensus.cc:504] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7f0739070469684bd2bba9de15ff3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45001 } } peers { permanent_uuid: "a236ca713225496e87549e9b6f2e8251" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 40727 } } peers { permanent_uuid: "47e633bcc1bd4d89affc004b36b7dad7" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 33873 } }
I20260806 13:10:25.622037 8187 leader_election.cc:288] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00d7f0739070469684bd2bba9de15ff3 (127.13.224.67:45001), a236ca713225496e87549e9b6f2e8251 (127.13.224.66:40727)
I20260806 13:10:25.624701 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.622242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:33261 (local address 127.13.224.66:40727)
0806 13:10:25.622285 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:25.622287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:25.622296 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:10:25.622371 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:25.622372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:25.622383 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:25.622436 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:25.622587 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.623082 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.624005 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.624385 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.624451 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.624458 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:25.624595 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 13:10:25.624632 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:10:25.624703 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.622289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:37313 (local address 127.13.224.67:45001)
0806 13:10:25.622329 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:25.622331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:25.622433 (+ 102us) server_negotiation.cc:407] Connection header received
0806 13:10:25.622483 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:25.622484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:25.622493 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:25.622546 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:25.622687 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.623129 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.624027 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.624384 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.624444 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.624453 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:25.624600 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 13:10:25.624638 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:10:25.624717 7758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.622190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:40727 (local address 127.13.224.65:33261)
0806 13:10:25.622241 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:25.622251 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:25.622291 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:25.622446 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:25.622447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:25.622448 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:25.622522 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.622524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.623150 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.623151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.623918 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.623921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.624467 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.624468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.624529 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.624532 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:10:25.624576 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 13:10:25.624617 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":25}
I20260806 13:10:25.624727 8191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.622205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:45001 (local address 127.13.224.65:37313)
0806 13:10:25.622358 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:25.622368 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:25.622430 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:25.622548 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:25.622548 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:25.622550 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:25.622630 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.622631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.623201 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.623202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.623943 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:25.623945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.624462 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.624463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:25.624535 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.624538 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:10:25.624585 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 13:10:25.624631 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260806 13:10:25.625053 7892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ee7fb27eb0407baa676657da0845fe" candidate_uuid: "47e633bcc1bd4d89affc004b36b7dad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a236ca713225496e87549e9b6f2e8251" is_pre_election: true
I20260806 13:10:25.625205 7892 raft_consensus.cc:2454] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47e633bcc1bd4d89affc004b36b7dad7 in term 0.
I20260806 13:10:25.625188 8084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ee7fb27eb0407baa676657da0845fe" candidate_uuid: "47e633bcc1bd4d89affc004b36b7dad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d7f0739070469684bd2bba9de15ff3" is_pre_election: true
I20260806 13:10:25.625327 8084 raft_consensus.cc:2454] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47e633bcc1bd4d89affc004b36b7dad7 in term 0.
I20260806 13:10:25.625416 7591 leader_election.cc:302] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47e633bcc1bd4d89affc004b36b7dad7, a236ca713225496e87549e9b6f2e8251; no voters:
I20260806 13:10:25.625531 8187 raft_consensus.cc:2781] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:10:25.625607 8187 raft_consensus.cc:482] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:10:25.625636 8187 raft_consensus.cc:3037] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:25.626374 8187 raft_consensus.cc:504] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7f0739070469684bd2bba9de15ff3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45001 } } peers { permanent_uuid: "a236ca713225496e87549e9b6f2e8251" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 40727 } } peers { permanent_uuid: "47e633bcc1bd4d89affc004b36b7dad7" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 33873 } }
I20260806 13:10:25.626497 8187 leader_election.cc:288] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [CANDIDATE]: Term 1 election: Requested vote from peers 00d7f0739070469684bd2bba9de15ff3 (127.13.224.67:45001), a236ca713225496e87549e9b6f2e8251 (127.13.224.66:40727)
I20260806 13:10:25.626662 8084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ee7fb27eb0407baa676657da0845fe" candidate_uuid: "47e633bcc1bd4d89affc004b36b7dad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d7f0739070469684bd2bba9de15ff3"
I20260806 13:10:25.626749 8084 raft_consensus.cc:3037] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:25.627667 8084 raft_consensus.cc:2454] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47e633bcc1bd4d89affc004b36b7dad7 in term 1.
I20260806 13:10:25.627876 7589 leader_election.cc:302] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d7f0739070469684bd2bba9de15ff3, 47e633bcc1bd4d89affc004b36b7dad7; no voters:
I20260806 13:10:25.627959 8187 raft_consensus.cc:2781] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:25.628082 8187 raft_consensus.cc:686] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [term 1 LEADER]: Becoming Leader. State: Replica: 47e633bcc1bd4d89affc004b36b7dad7, State: Running, Role: LEADER
I20260806 13:10:25.628180 8187 consensus_queue.cc:237] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7f0739070469684bd2bba9de15ff3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45001 } } peers { permanent_uuid: "a236ca713225496e87549e9b6f2e8251" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 40727 } } peers { permanent_uuid: "47e633bcc1bd4d89affc004b36b7dad7" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 33873 } }
I20260806 13:10:25.629997 7892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ee7fb27eb0407baa676657da0845fe" candidate_uuid: "47e633bcc1bd4d89affc004b36b7dad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a236ca713225496e87549e9b6f2e8251"
I20260806 13:10:25.630134 7892 raft_consensus.cc:3037] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:25.630141 7478 catalog_manager.cc:5495] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47e633bcc1bd4d89affc004b36b7dad7 (127.13.224.65). New cstate: current_term: 1 leader_uuid: "47e633bcc1bd4d89affc004b36b7dad7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7f0739070469684bd2bba9de15ff3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a236ca713225496e87549e9b6f2e8251" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 40727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47e633bcc1bd4d89affc004b36b7dad7" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 33873 } health_report { overall_health: HEALTHY } } }
I20260806 13:10:25.630991 7892 raft_consensus.cc:2454] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47e633bcc1bd4d89affc004b36b7dad7 in term 1.
I20260806 13:10:25.636260 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.634178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56410 (local address 127.13.224.65:33873)
0806 13:10:25.634214 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:25.634216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:25.634224 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:10:25.634289 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:25.634290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:25.634306 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:25.634385 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:25.634460 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.635070 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.635742 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:25.635914 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:25.635962 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:25.636047 (+ 85us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:25.636141 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:25.636192 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:25.636204 (+ 12us) server_negotiation.cc:299] Negotiation successful
0806 13:10:25.636217 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:10:25.637717 7892 raft_consensus.cc:1261] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251 [term 1 FOLLOWER]: Refusing update from remote peer 47e633bcc1bd4d89affc004b36b7dad7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:25.637751 8084 raft_consensus.cc:1261] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3 [term 1 FOLLOWER]: Refusing update from remote peer 47e633bcc1bd4d89affc004b36b7dad7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:25.638060 8193 consensus_queue.cc:1035] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00d7f0739070469684bd2bba9de15ff3" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 45001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:25.638175 8187 consensus_queue.cc:1035] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a236ca713225496e87549e9b6f2e8251" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:25.640556 8197 mvcc.cc:205] Tried to move back new op lower bound from 7315545397810044928 to 7315545397773541376. Current Snapshot: MvccSnapshot[applied={T|T < 7315545397810044928}]
I20260806 13:10:25.642761 7478 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:10:25.642835 7478 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:10:25.643438 7478 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3451015d2e364c05aa4601e200584908: Sending DeleteTablet for 3 replicas of tablet 94ee7fb27eb0407baa676657da0845fe
I20260806 13:10:25.643746 7665 tablet_service.cc:1514] Processing DeleteTablet for tablet 94ee7fb27eb0407baa676657da0845fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:25 UTC) from {username='slave'} at 127.0.0.1:56400
I20260806 13:10:25.643756 8051 tablet_service.cc:1514] Processing DeleteTablet for tablet 94ee7fb27eb0407baa676657da0845fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:25 UTC) from {username='slave'} at 127.0.0.1:37734
I20260806 13:10:25.643821 7858 tablet_service.cc:1514] Processing DeleteTablet for tablet 94ee7fb27eb0407baa676657da0845fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:25 UTC) from {username='slave'} at 127.0.0.1:60192
I20260806 13:10:25.644002 8217 tablet_replica.cc:330] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3: stopping tablet replica
I20260806 13:10:25.644054 8218 tablet_replica.cc:330] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251: stopping tablet replica
I20260806 13:10:25.644174 8218 raft_consensus.cc:2229] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:10:25.644210 8217 raft_consensus.cc:2229] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:10:25.644304 8218 raft_consensus.cc:2258] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:25.644302 8215 tablet_replica.cc:330] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7: stopping tablet replica
I20260806 13:10:25.644342 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 7567
I20260806 13:10:25.644436 8215 raft_consensus.cc:2229] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:10:25.644450 8217 raft_consensus.cc:2258] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:25.644569 8215 raft_consensus.cc:2258] T 94ee7fb27eb0407baa676657da0845fe P 47e633bcc1bd4d89affc004b36b7dad7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:25.647113 8218 ts_tablet_manager.cc:1905] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:10:25.651628 8217 ts_tablet_manager.cc:1905] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:10:25.653146 8217 ts_tablet_manager.cc:1918] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:10:25.653218 8217 log.cc:1196] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/94ee7fb27eb0407baa676657da0845fe
I20260806 13:10:25.653452 8217 ts_tablet_manager.cc:1939] T 94ee7fb27eb0407baa676657da0845fe P 00d7f0739070469684bd2bba9de15ff3: Deleting consensus metadata
I20260806 13:10:25.653944 7452 catalog_manager.cc:4841] TS 00d7f0739070469684bd2bba9de15ff3 (127.13.224.67:45001): tablet 94ee7fb27eb0407baa676657da0845fe (table test-table [id=59c62c1e11a84cb49257b7c79e2f7141]) successfully deleted
I20260806 13:10:25.655105 8218 ts_tablet_manager.cc:1918] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:10:25.655207 8218 log.cc:1196] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/94ee7fb27eb0407baa676657da0845fe
I20260806 13:10:25.655444 8218 ts_tablet_manager.cc:1939] T 94ee7fb27eb0407baa676657da0845fe P a236ca713225496e87549e9b6f2e8251: Deleting consensus metadata
I20260806 13:10:25.655822 7459 catalog_manager.cc:4841] TS a236ca713225496e87549e9b6f2e8251 (127.13.224.66:40727): tablet 94ee7fb27eb0407baa676657da0845fe (table test-table [id=59c62c1e11a84cb49257b7c79e2f7141]) successfully deleted
W20260806 13:10:25.688311 7456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:33873 (error 108)
I20260806 13:10:25.688565 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 7759
I20260806 13:10:25.690442 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:25.688685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:33873 (local address 127.0.0.1:56424)
0806 13:10:25.690363 (+ 1678us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:25.690410 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1654}
W20260806 13:10:25.698779 7456 catalog_manager.cc:4584] TS 47e633bcc1bd4d89affc004b36b7dad7 (127.13.224.65:33873): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94ee7fb27eb0407baa676657da0845fe: Network error: Client connection negotiation failed: client connection to 127.13.224.65:33873: connect: Connection refused (error 111)
I20260806 13:10:25.709860 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 7966
I20260806 13:10:25.728600 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 7427
2026-08-06T13:10:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (802 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:25 dist-test-slave-q9h9 krb5kdc[8223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:25 dist-test-slave-q9h9 krb5kdc[8223](info): set up 2 sockets
Aug 06 13:10:25 dist-test-slave-q9h9 krb5kdc[8223](info): commencing operation
krb5kdc: starting...
W20260806 13:10:26.825518 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:26 dist-test-slave-q9h9 krb5kdc[8223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:10:26.930856 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:43205
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:43205
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:27.009569 8240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:27.009749 8240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:27.009779 8240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:27.010890 8240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:27.010931 8240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:27.010949 8240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:27.010967 8240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:27.010982 8240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:27.012400 8240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:43205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:43205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:27.012626 8240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:27.012810 8240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:27.017195 8249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:27.017228 8252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:27.017253 8240 server_base.cc:1034] running on GCE node
W20260806 13:10:27.017215 8250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:27.017586 8240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:27.017872 8240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:27.018998 8240 hybrid_clock.cc:648] HybridClock initialized: now 1786021827018991 us; error 27 us; skew 500 ppm
Aug 06 13:10:27 dist-test-slave-q9h9 krb5kdc[8223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021827, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:27.021106 8240 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:10:27.021486 8240 webserver.cc:460] Webserver started at http://127.13.224.126:41305/ using document root <none> and password file <none>
I20260806 13:10:27.021677 8240 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:27.021723 8240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:27.021842 8240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:27.022806 8240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "89781d41d3e1447f8f8efe445ed8b22b"
format_stamp: "Formatted at 2026-08-06 13:10:27 on dist-test-slave-q9h9"
server_key: "ef732b35b038137294f37f199adcf5bc"
server_key_iv: "876af040256cf6b1b2fa8f5ef696a8c9"
server_key_version: "encryptionkey@0"
I20260806 13:10:27.023119 8240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "89781d41d3e1447f8f8efe445ed8b22b"
format_stamp: "Formatted at 2026-08-06 13:10:27 on dist-test-slave-q9h9"
server_key: "ef732b35b038137294f37f199adcf5bc"
server_key_iv: "876af040256cf6b1b2fa8f5ef696a8c9"
server_key_version: "encryptionkey@0"
I20260806 13:10:27.024282 8240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:27.024905 8260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:27.025048 8240 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:27.025130 8240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "89781d41d3e1447f8f8efe445ed8b22b"
format_stamp: "Formatted at 2026-08-06 13:10:27 on dist-test-slave-q9h9"
server_key: "ef732b35b038137294f37f199adcf5bc"
server_key_iv: "876af040256cf6b1b2fa8f5ef696a8c9"
server_key_version: "encryptionkey@0"
I20260806 13:10:27.025194 8240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:27.044428 8240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:27.046075 8240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:27.046200 8240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:27.049806 8240 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:43205
I20260806 13:10:27.049839 8342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:43205 every 8 connection(s)
I20260806 13:10:27.050140 8240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:27.050866 8344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:27.052883 8344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b: Bootstrap starting.
I20260806 13:10:27.053417 8344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:27.053665 8344 log.cc:824] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:27.054312 8344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b: No bootstrap required, opened a new log
I20260806 13:10:27.054642 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 8240
I20260806 13:10:27.054734 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:27.054914 14209 external_mini_cluster.cc:1428] Setting key c559011f9a123958bed95533b0f6df96
I20260806 13:10:27.056223 8344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89781d41d3e1447f8f8efe445ed8b22b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43205 } }
I20260806 13:10:27.056380 8344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:27.056416 8344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89781d41d3e1447f8f8efe445ed8b22b, State: Initialized, Role: FOLLOWER
I20260806 13:10:27.056536 8344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89781d41d3e1447f8f8efe445ed8b22b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43205 } }
I20260806 13:10:27.056608 8344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:27.056629 8344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:27.056682 8344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:27.057734 8344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89781d41d3e1447f8f8efe445ed8b22b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43205 } }
I20260806 13:10:27.057842 8344 leader_election.cc:302] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89781d41d3e1447f8f8efe445ed8b22b; no voters:
I20260806 13:10:27.057989 8344 leader_election.cc:288] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:27.058048 8350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:27.058179 8350 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [term 1 LEADER]: Becoming Leader. State: Replica: 89781d41d3e1447f8f8efe445ed8b22b, State: Running, Role: LEADER
I20260806 13:10:27.058256 8344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:27.058261 8350 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89781d41d3e1447f8f8efe445ed8b22b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43205 } }
I20260806 13:10:27.058789 8351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89781d41d3e1447f8f8efe445ed8b22b. Latest consensus state: current_term: 1 leader_uuid: "89781d41d3e1447f8f8efe445ed8b22b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89781d41d3e1447f8f8efe445ed8b22b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43205 } } }
I20260806 13:10:27.058874 8351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:27.058794 8353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89781d41d3e1447f8f8efe445ed8b22b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89781d41d3e1447f8f8efe445ed8b22b" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43205 } } }
I20260806 13:10:27.059069 8353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:27.059152 8364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:27.059657 8364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:27.061116 8364 catalog_manager.cc:1347] Generated new cluster ID: ee2f61716e10471e8f898f2965cf5375
I20260806 13:10:27.061169 8364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:10:27 dist-test-slave-q9h9 krb5kdc[8223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:27.066118 8349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.055358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.13.224.126:43205)
0806 13:10:27.055551 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:27.055560 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:27.055593 (+ 33us) server_negotiation.cc:407] Connection header received
0806 13:10:27.056193 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:27.056202 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:27.056263 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:27.056405 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:27.057076 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.058399 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.059515 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.060083 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.060172 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.062731 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:27.062734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:27.062735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:27.062746 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:27.064565 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:27.064925 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:27.064927 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:27.064928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:27.065062 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:27.065236 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:27.065237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:27.065237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:27.065523 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:27.065692 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:27.065759 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 13:10:27.065886 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
I20260806 13:10:27.070130 8364 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:27.070595 8364 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:27.076229 8364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b: Generated new TSK 0
I20260806 13:10:27.076378 8364 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:10:27.105409 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43205
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:27.184274 8383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:27.184475 8383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:27.184507 8383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:27.185936 8383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:27.185984 8383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:27.186044 8383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:10:27.187489 8383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:27.187702 8383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:27.187908 8383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:27.192538 8393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:27.192587 8392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:27.192633 8383 server_base.cc:1034] running on GCE node
W20260806 13:10:27.192546 8395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:27.192924 8383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:27.193147 8383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:27.194286 8383 hybrid_clock.cc:648] HybridClock initialized: now 1786021827194274 us; error 29 us; skew 500 ppm
Aug 06 13:10:27 dist-test-slave-q9h9 krb5kdc[8223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021827, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:27.196503 8383 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:10:27.196888 8383 webserver.cc:460] Webserver started at http://127.13.224.65:35529/ using document root <none> and password file <none>
I20260806 13:10:27.197080 8383 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:27.197129 8383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:27.197252 8383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:27.198241 8383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "6a984459dc7544ce8cbda3ca1ba204aa"
format_stamp: "Formatted at 2026-08-06 13:10:27 on dist-test-slave-q9h9"
server_key: "fe6dc59017a126da84e590105881a602"
server_key_iv: "1744c5d52326ebd5c9015d6f0c5bee92"
server_key_version: "encryptionkey@0"
I20260806 13:10:27.198549 8383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "6a984459dc7544ce8cbda3ca1ba204aa"
format_stamp: "Formatted at 2026-08-06 13:10:27 on dist-test-slave-q9h9"
server_key: "fe6dc59017a126da84e590105881a602"
server_key_iv: "1744c5d52326ebd5c9015d6f0c5bee92"
server_key_version: "encryptionkey@0"
I20260806 13:10:27.199724 8383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:10:27.200352 8406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:27.200491 8383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:27.200573 8383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "6a984459dc7544ce8cbda3ca1ba204aa"
format_stamp: "Formatted at 2026-08-06 13:10:27 on dist-test-slave-q9h9"
server_key: "fe6dc59017a126da84e590105881a602"
server_key_iv: "1744c5d52326ebd5c9015d6f0c5bee92"
server_key_version: "encryptionkey@0"
I20260806 13:10:27.200635 8383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:27.208809 8383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:27.210379 8383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:27.210500 8383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:27.210693 8383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:27.210960 8383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:27.211000 8383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:27.211027 8383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:27.211051 8383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:27.216461 8383 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:40313
I20260806 13:10:27.216503 8583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:40313 every 8 connection(s)
I20260806 13:10:27.216770 8383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:10:27.219897 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 8383
I20260806 13:10:27.220006 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:10:27.220212 14209 external_mini_cluster.cc:1428] Setting key d447efba3d8b0cf0aecfba3a72ab8c28
Aug 06 13:10:27 dist-test-slave-q9h9 krb5kdc[8223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021827, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:27.225795 8349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.217352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:56779 (local address 127.13.224.126:43205)
0806 13:10:27.217403 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:27.217405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:27.217602 (+ 197us) server_negotiation.cc:407] Connection header received
0806 13:10:27.218050 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:27.218051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:27.218065 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:27.218127 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:27.218938 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.219421 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.220616 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.220900 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.220959 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.223143 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:27.223144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:27.223145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:27.223154 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:27.224368 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:27.224793 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:27.224793 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:27.224794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:27.224840 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:27.225150 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:27.225151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:27.225152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:27.225283 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:27.225348 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:27.225659 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 13:10:27.225733 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 13:10:27.225987 8587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.217337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:43205 (local address 127.13.224.65:56779)
0806 13:10:27.217495 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:27.217532 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:27.217937 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:27.218182 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:27.218186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:27.218686 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:27.218871 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.218875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.219478 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.219478 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.220543 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.220546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.220980 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.220981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.221123 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.221532 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:27.221541 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:27.223057 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:27.224449 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:27.224457 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:27.224459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:27.224721 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:27.224896 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:27.224897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:27.224897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:27.225030 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:27.225353 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:27.225359 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:27.225511 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 13:10:27.225624 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 13:10:27.225777 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260806 13:10:27.226522 8584 heartbeater.cc:347] Connected to a master server at 127.13.224.126:43205
I20260806 13:10:27.226605 8584 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:27.226778 8584 heartbeater.cc:511] Master 127.13.224.126:43205 requested a full tablet report, sending...
I20260806 13:10:27.227283 8288 ts_manager.cc:194] Registered new tserver with Master: 6a984459dc7544ce8cbda3ca1ba204aa (127.13.224.65:40313)
I20260806 13:10:27.228197 8288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:56779
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:10:27.243858 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43205
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:27.322101 8593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:27.322274 8593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:27.322295 8593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:27.323352 8593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:27.323382 8593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:27.323405 8593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:10:27.324762 8593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:27.324929 8593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:27.325081 8593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:27.329562 8599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:27.329587 8600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:27.329630 8593 server_base.cc:1034] running on GCE node
W20260806 13:10:27.329563 8604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:27.330093 8593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:27.330319 8593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:27.331450 8593 hybrid_clock.cc:648] HybridClock initialized: now 1786021827331439 us; error 30 us; skew 500 ppm
Aug 06 13:10:27 dist-test-slave-q9h9 krb5kdc[8223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021827, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:27.333614 8593 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:10:27.333993 8593 webserver.cc:460] Webserver started at http://127.13.224.66:38815/ using document root <none> and password file <none>
I20260806 13:10:27.334205 8593 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:27.334251 8593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:27.334368 8593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:27.335343 8593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "6cf9a6d579ac4a84b84dde8945d868e7"
format_stamp: "Formatted at 2026-08-06 13:10:27 on dist-test-slave-q9h9"
server_key: "8f512b5c991c043373e5e4660a6894dc"
server_key_iv: "5511fafb045b0178d912f30efa9aa72e"
server_key_version: "encryptionkey@0"
I20260806 13:10:27.335665 8593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "6cf9a6d579ac4a84b84dde8945d868e7"
format_stamp: "Formatted at 2026-08-06 13:10:27 on dist-test-slave-q9h9"
server_key: "8f512b5c991c043373e5e4660a6894dc"
server_key_iv: "5511fafb045b0178d912f30efa9aa72e"
server_key_version: "encryptionkey@0"
I20260806 13:10:27.336810 8593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:10:27.337404 8613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:27.337575 8593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:27.337659 8593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "6cf9a6d579ac4a84b84dde8945d868e7"
format_stamp: "Formatted at 2026-08-06 13:10:27 on dist-test-slave-q9h9"
server_key: "8f512b5c991c043373e5e4660a6894dc"
server_key_iv: "5511fafb045b0178d912f30efa9aa72e"
server_key_version: "encryptionkey@0"
I20260806 13:10:27.337726 8593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:27.347785 8593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:27.349365 8593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:27.349483 8593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:27.349673 8593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:27.349928 8593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:27.349960 8593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:27.349978 8593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:27.350003 8593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:27.355394 8593 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:39887
I20260806 13:10:27.355434 8800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:39887 every 8 connection(s)
I20260806 13:10:27.355676 8593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:10:27.357887 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 8593
I20260806 13:10:27.357995 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:10:27.358203 14209 external_mini_cluster.cc:1428] Setting key a57b0176b3362e1959cfce4c2042bef6
Aug 06 13:10:27 dist-test-slave-q9h9 krb5kdc[8223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021827, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:27.364382 8349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.356267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:53821 (local address 127.13.224.126:43205)
0806 13:10:27.356318 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:27.356320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:27.356501 (+ 181us) server_negotiation.cc:407] Connection header received
0806 13:10:27.356875 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:27.356876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:27.356885 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:27.356930 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:27.357722 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.358465 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.359537 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.359794 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.359862 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.361912 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:27.361913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:27.361913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:27.361919 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:27.363172 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:27.363522 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:27.363523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:27.363523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:27.363575 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:27.363788 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:27.363789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:27.363790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:27.363892 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:27.363948 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:27.364251 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 13:10:27.364319 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 13:10:27.364553 8804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.356240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:43205 (local address 127.13.224.66:53821)
0806 13:10:27.356399 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:27.356436 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:27.356809 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:27.356981 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:27.356984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:27.357482 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:27.357651 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.357655 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.358535 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.358536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.359462 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.359465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.359864 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.359864 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.359955 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.360369 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:27.360377 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:27.361829 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:27.363245 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:27.363251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:27.363253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:27.363458 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:27.363630 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:27.363631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:27.363631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:27.363732 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:27.363946 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:27.363953 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:27.364097 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 13:10:27.364203 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 13:10:27.364354 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260806 13:10:27.365048 8802 heartbeater.cc:347] Connected to a master server at 127.13.224.126:43205
I20260806 13:10:27.365139 8802 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:27.365314 8802 heartbeater.cc:511] Master 127.13.224.126:43205 requested a full tablet report, sending...
I20260806 13:10:27.365638 8288 ts_manager.cc:194] Registered new tserver with Master: 6cf9a6d579ac4a84b84dde8945d868e7 (127.13.224.66:39887)
I20260806 13:10:27.365986 8288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:53821
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:10:27.382472 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43205
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:27.460335 8811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:27.460502 8811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:27.460523 8811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:27.461606 8811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:27.461644 8811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:27.461670 8811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:10:27.463081 8811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:27.463284 8811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:27.463474 8811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:27.467962 8822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:27.467917 8820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:27.467923 8819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:27.468139 8811 server_base.cc:1034] running on GCE node
I20260806 13:10:27.468322 8811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:27.468497 8811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:27.469615 8811 hybrid_clock.cc:648] HybridClock initialized: now 1786021827469608 us; error 27 us; skew 500 ppm
Aug 06 13:10:27 dist-test-slave-q9h9 krb5kdc[8223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021827, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:27.471851 8811 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:10:27.472229 8811 webserver.cc:460] Webserver started at http://127.13.224.67:41203/ using document root <none> and password file <none>
I20260806 13:10:27.472411 8811 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:27.472450 8811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:27.472535 8811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:27.473400 8811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "25bc640efa60424099cc8be5198f2b54"
format_stamp: "Formatted at 2026-08-06 13:10:27 on dist-test-slave-q9h9"
server_key: "986a06ebc5833bc3f0d09862beac7f89"
server_key_iv: "1cab574584e1789f84575aac0eddad43"
server_key_version: "encryptionkey@0"
I20260806 13:10:27.473666 8811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "25bc640efa60424099cc8be5198f2b54"
format_stamp: "Formatted at 2026-08-06 13:10:27 on dist-test-slave-q9h9"
server_key: "986a06ebc5833bc3f0d09862beac7f89"
server_key_iv: "1cab574584e1789f84575aac0eddad43"
server_key_version: "encryptionkey@0"
I20260806 13:10:27.474851 8811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:10:27.475476 8832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:27.475623 8811 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:10:27.475692 8811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "25bc640efa60424099cc8be5198f2b54"
format_stamp: "Formatted at 2026-08-06 13:10:27 on dist-test-slave-q9h9"
server_key: "986a06ebc5833bc3f0d09862beac7f89"
server_key_iv: "1cab574584e1789f84575aac0eddad43"
server_key_version: "encryptionkey@0"
I20260806 13:10:27.475750 8811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:27.488602 8811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:27.490497 8811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:27.490622 8811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:27.490810 8811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:27.491066 8811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:27.491098 8811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:27.491123 8811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:27.491142 8811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:27.496477 8811 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:32883
I20260806 13:10:27.496515 8998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:32883 every 8 connection(s)
I20260806 13:10:27.496768 8811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
Aug 06 13:10:27 dist-test-slave-q9h9 krb5kdc[8223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021827, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:27.504493 8349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.497364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:46221 (local address 127.13.224.126:43205)
0806 13:10:27.497408 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:27.497410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:27.497589 (+ 179us) server_negotiation.cc:407] Connection header received
0806 13:10:27.498032 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:27.498034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:27.498045 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:27.498123 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:27.498930 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.499426 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.500288 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.500462 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.500515 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.502335 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:27.502337 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:27.502337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:27.502345 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:27.503429 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:27.503747 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:27.503748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:27.503748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:27.503792 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:27.503945 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:27.503945 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:27.503946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:27.504035 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:27.504078 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:27.504317 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 13:10:27.504404 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:10:27.504616 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.497331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:43205 (local address 127.13.224.67:46221)
0806 13:10:27.497479 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:27.497519 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:27.497938 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:27.498180 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:27.498183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:27.498685 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:27.498862 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.498866 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.499484 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.499485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.500226 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.500228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.500522 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.500522 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.500610 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.500956 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:27.500963 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:27.502244 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:27.503489 (+ 1245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:27.503495 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:27.503496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:27.503690 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:27.503827 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:27.503828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:27.503828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:27.503906 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:27.504075 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:27.504078 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:27.504172 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 13:10:27.504274 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 13:10:27.504433 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260806 13:10:27.505028 9000 heartbeater.cc:347] Connected to a master server at 127.13.224.126:43205
I20260806 13:10:27.505116 9000 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:27.505277 9000 heartbeater.cc:511] Master 127.13.224.126:43205 requested a full tablet report, sending...
I20260806 13:10:27.505586 8288 ts_manager.cc:194] Registered new tserver with Master: 25bc640efa60424099cc8be5198f2b54 (127.13.224.67:32883)
I20260806 13:10:27.505913 8288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:46221
I20260806 13:10:27.506421 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 8811
I20260806 13:10:27.506539 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:10:27.506701 14209 external_mini_cluster.cc:1428] Setting key b2402cc1efa911e9dafab248948655a3
I20260806 13:10:27.507403 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:10:27.513787 8349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.508615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34574 (local address 127.13.224.126:43205)
0806 13:10:27.508669 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:27.508671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:27.508787 (+ 116us) server_negotiation.cc:407] Connection header received
0806 13:10:27.508812 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:27.508813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:27.508823 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:27.508886 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:27.509361 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.509813 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.510504 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.510688 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.510741 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.511798 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:27.511799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:27.511800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:27.511805 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:27.512898 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:27.513219 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:27.513220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:27.513221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:27.513273 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:27.513419 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:27.513420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:27.513420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:27.513526 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:27.513595 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:27.513683 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 13:10:27.513730 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 13:10:27.515101 8288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:10:27.516510 8288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:10:27.522555 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.518638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59898 (local address 127.13.224.67:32883)
0806 13:10:27.518803 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:27.518807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:27.518817 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:10:27.518848 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:27.518850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:27.518863 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:27.518931 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:27.519089 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.520118 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.521531 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.522195 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.522264 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.522309 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:27.522451 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 13:10:27.522488 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260806 13:10:27.522564 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.518317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:32883 (local address 127.0.0.1:59898)
0806 13:10:27.518434 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:27.518462 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:27.518526 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:27.518948 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:27.518951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:27.518953 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:27.519010 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.519012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.520219 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.520220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.521447 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.521450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.522255 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.522256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.522335 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.522348 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 13:10:27.522415 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 13:10:27.522476 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":51,"thread_start_us":77,"threads_started":1}
I20260806 13:10:27.522789 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.518622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52524 (local address 127.13.224.66:39887)
0806 13:10:27.518811 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:27.518814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:27.519038 (+ 224us) server_negotiation.cc:407] Connection header received
0806 13:10:27.519152 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:27.519154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:27.519182 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:27.519249 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:27.519426 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.520398 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.521415 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.522158 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.522231 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.522290 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:27.522657 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 13:10:27.522701 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260806 13:10:27.523329 9013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.518306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:40313 (local address 127.0.0.1:40128)
0806 13:10:27.518425 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:27.518462 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:27.518526 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:27.519127 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:27.519128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:27.519130 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:27.519180 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.519182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.520117 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.520118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.521447 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.521450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.523090 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.523091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.523152 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.523165 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 13:10:27.523227 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 13:10:27.523264 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":36,"thread_start_us":69,"threads_started":1}
I20260806 13:10:27.523362 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.518807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40128 (local address 127.13.224.65:40313)
0806 13:10:27.518957 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:27.518961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:27.518972 (+ 11us) server_negotiation.cc:407] Connection header received
0806 13:10:27.519015 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:27.519017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:27.519032 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:27.519256 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:27.519336 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.520024 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.522347 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.523017 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.523073 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.523108 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:27.523241 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 13:10:27.523276 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260806 13:10:27.523453 8909 tablet_service.cc:1467] Processing CreateTablet for tablet 5b59d3ba62c54433b7f708d09c6524ff (DEFAULT_TABLE table=test-table [id=94721309fd2f47ffafe18c171d6b5c61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:27.523777 8909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b59d3ba62c54433b7f708d09c6524ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:27.524216 8478 tablet_service.cc:1467] Processing CreateTablet for tablet 5b59d3ba62c54433b7f708d09c6524ff (DEFAULT_TABLE table=test-table [id=94721309fd2f47ffafe18c171d6b5c61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:27.524545 8478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b59d3ba62c54433b7f708d09c6524ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:27.526908 9025 tablet_bootstrap.cc:492] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54: Bootstrap starting.
I20260806 13:10:27.522564 9015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.518504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:39887 (local address 127.0.0.1:52524)
0806 13:10:27.518972 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:27.518982 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:27.519041 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:27.519328 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:27.519329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:27.519331 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:27.519362 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.519364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.520485 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.520486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.521345 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.521348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.522243 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.522244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.522335 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.522348 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 13:10:27.522418 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 13:10:27.522476 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":22,"thread_start_us":81,"threads_started":1}
I20260806 13:10:27.527519 9025 tablet_bootstrap.cc:654] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:27.527827 8696 tablet_service.cc:1467] Processing CreateTablet for tablet 5b59d3ba62c54433b7f708d09c6524ff (DEFAULT_TABLE table=test-table [id=94721309fd2f47ffafe18c171d6b5c61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:27.528069 9025 log.cc:824] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:27.528087 8696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b59d3ba62c54433b7f708d09c6524ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:27.528728 9025 tablet_bootstrap.cc:492] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54: No bootstrap required, opened a new log
I20260806 13:10:27.528787 9025 ts_tablet_manager.cc:1397] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:10:27.530206 9026 tablet_bootstrap.cc:492] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa: Bootstrap starting.
I20260806 13:10:27.530324 9025 raft_consensus.cc:348] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a984459dc7544ce8cbda3ca1ba204aa" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40313 } } peers { permanent_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39887 } } peers { permanent_uuid: "25bc640efa60424099cc8be5198f2b54" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 32883 } }
I20260806 13:10:27.530470 9025 raft_consensus.cc:374] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:27.530508 9025 raft_consensus.cc:729] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25bc640efa60424099cc8be5198f2b54, State: Initialized, Role: FOLLOWER
I20260806 13:10:27.530633 9025 consensus_queue.cc:260] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a984459dc7544ce8cbda3ca1ba204aa" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40313 } } peers { permanent_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39887 } } peers { permanent_uuid: "25bc640efa60424099cc8be5198f2b54" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 32883 } }
I20260806 13:10:27.530840 9026 tablet_bootstrap.cc:654] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:27.530900 9025 ts_tablet_manager.cc:1428] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:10:27.530926 9000 heartbeater.cc:503] Master 127.13.224.126:43205 was elected leader, sending a full tablet report...
I20260806 13:10:27.531078 9029 tablet_bootstrap.cc:492] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7: Bootstrap starting.
I20260806 13:10:27.531383 9026 log.cc:824] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:27.531672 9029 tablet_bootstrap.cc:654] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:27.532168 9029 log.cc:824] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:27.532243 9026 tablet_bootstrap.cc:492] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa: No bootstrap required, opened a new log
I20260806 13:10:27.532306 9026 ts_tablet_manager.cc:1397] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:10:27.532757 9029 tablet_bootstrap.cc:492] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7: No bootstrap required, opened a new log
I20260806 13:10:27.532809 9029 ts_tablet_manager.cc:1397] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:10:27.533604 9026 raft_consensus.cc:348] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a984459dc7544ce8cbda3ca1ba204aa" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40313 } } peers { permanent_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39887 } } peers { permanent_uuid: "25bc640efa60424099cc8be5198f2b54" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 32883 } }
I20260806 13:10:27.533739 9026 raft_consensus.cc:374] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:27.533772 9026 raft_consensus.cc:729] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a984459dc7544ce8cbda3ca1ba204aa, State: Initialized, Role: FOLLOWER
I20260806 13:10:27.533876 9026 consensus_queue.cc:260] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a984459dc7544ce8cbda3ca1ba204aa" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40313 } } peers { permanent_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39887 } } peers { permanent_uuid: "25bc640efa60424099cc8be5198f2b54" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 32883 } }
I20260806 13:10:27.534154 9026 ts_tablet_manager.cc:1428] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 13:10:27.534257 8584 heartbeater.cc:503] Master 127.13.224.126:43205 was elected leader, sending a full tablet report...
I20260806 13:10:27.534241 9029 raft_consensus.cc:348] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a984459dc7544ce8cbda3ca1ba204aa" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40313 } } peers { permanent_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39887 } } peers { permanent_uuid: "25bc640efa60424099cc8be5198f2b54" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 32883 } }
I20260806 13:10:27.534420 9029 raft_consensus.cc:374] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:27.534458 9029 raft_consensus.cc:729] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cf9a6d579ac4a84b84dde8945d868e7, State: Initialized, Role: FOLLOWER
I20260806 13:10:27.534564 9029 consensus_queue.cc:260] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a984459dc7544ce8cbda3ca1ba204aa" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40313 } } peers { permanent_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39887 } } peers { permanent_uuid: "25bc640efa60424099cc8be5198f2b54" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 32883 } }
I20260806 13:10:27.534834 9029 ts_tablet_manager.cc:1428] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:10:27.535022 8802 heartbeater.cc:503] Master 127.13.224.126:43205 was elected leader, sending a full tablet report...
W20260806 13:10:27.606142 8803 tablet.cc:2369] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 13:10:27.717418 8585 tablet.cc:2369] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 13:10:27.747272 9002 tablet.cc:2369] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:10:27.881580 9038 raft_consensus.cc:482] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:10:27.881695 9038 raft_consensus.cc:504] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a984459dc7544ce8cbda3ca1ba204aa" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40313 } } peers { permanent_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39887 } } peers { permanent_uuid: "25bc640efa60424099cc8be5198f2b54" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 32883 } }
I20260806 13:10:27.882040 9038 leader_election.cc:288] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a984459dc7544ce8cbda3ca1ba204aa (127.13.224.65:40313), 25bc640efa60424099cc8be5198f2b54 (127.13.224.67:32883)
I20260806 13:10:27.885823 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.882239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:46449 (local address 127.13.224.65:40313)
0806 13:10:27.882282 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:27.882284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:27.882401 (+ 117us) server_negotiation.cc:407] Connection header received
0806 13:10:27.882457 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:27.882458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:27.882472 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:27.882521 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:27.882688 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.883428 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.884931 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.885469 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.885540 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.885549 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:27.885714 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 13:10:27.885757 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:10:27.885823 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.882294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:47809 (local address 127.13.224.67:32883)
0806 13:10:27.882350 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:27.882352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:27.882382 (+ 30us) server_negotiation.cc:407] Connection header received
0806 13:10:27.882457 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:27.882458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:27.882468 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:27.882521 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:27.882687 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.883436 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.884931 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.885489 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.885549 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.885556 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:27.885714 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 13:10:27.885749 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 13:10:27.885855 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.882177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:32883 (local address 127.13.224.66:47809)
0806 13:10:27.882327 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:27.882336 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:27.882397 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:27.882534 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:27.882534 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:27.882536 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:27.882621 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.882623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.883504 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.883504 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.884847 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.884851 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.885554 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.885555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.885647 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.885651 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:10:27.885702 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 13:10:27.885750 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":62,"thread_start_us":70,"threads_started":1}
I20260806 13:10:27.885859 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.882176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:40313 (local address 127.13.224.66:46449)
0806 13:10:27.882322 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:27.882332 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:27.882397 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:27.882534 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:27.882534 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:27.882537 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:27.882630 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.882631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.883503 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.883504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.884847 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:27.884851 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.885556 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.885557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:27.885650 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.885652 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:10:27.885702 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 13:10:27.885751 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":77,"thread_start_us":67,"threads_started":1}
I20260806 13:10:27.886229 8938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b59d3ba62c54433b7f708d09c6524ff" candidate_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25bc640efa60424099cc8be5198f2b54" is_pre_election: true
I20260806 13:10:27.886234 8512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b59d3ba62c54433b7f708d09c6524ff" candidate_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a984459dc7544ce8cbda3ca1ba204aa" is_pre_election: true
I20260806 13:10:27.886377 8512 raft_consensus.cc:2454] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cf9a6d579ac4a84b84dde8945d868e7 in term 0.
I20260806 13:10:27.886379 8938 raft_consensus.cc:2454] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cf9a6d579ac4a84b84dde8945d868e7 in term 0.
I20260806 13:10:27.886602 8618 leader_election.cc:302] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a984459dc7544ce8cbda3ca1ba204aa, 6cf9a6d579ac4a84b84dde8945d868e7; no voters:
I20260806 13:10:27.886765 9038 raft_consensus.cc:2781] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:10:27.886828 9038 raft_consensus.cc:482] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:10:27.886850 9038 raft_consensus.cc:3037] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:27.887527 9038 raft_consensus.cc:504] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a984459dc7544ce8cbda3ca1ba204aa" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40313 } } peers { permanent_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39887 } } peers { permanent_uuid: "25bc640efa60424099cc8be5198f2b54" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 32883 } }
I20260806 13:10:27.887643 9038 leader_election.cc:288] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [CANDIDATE]: Term 1 election: Requested vote from peers 6a984459dc7544ce8cbda3ca1ba204aa (127.13.224.65:40313), 25bc640efa60424099cc8be5198f2b54 (127.13.224.67:32883)
I20260806 13:10:27.887806 8512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b59d3ba62c54433b7f708d09c6524ff" candidate_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a984459dc7544ce8cbda3ca1ba204aa"
I20260806 13:10:27.887856 8938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b59d3ba62c54433b7f708d09c6524ff" candidate_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25bc640efa60424099cc8be5198f2b54"
I20260806 13:10:27.887878 8512 raft_consensus.cc:3037] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:27.887938 8938 raft_consensus.cc:3037] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:27.888549 8512 raft_consensus.cc:2454] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cf9a6d579ac4a84b84dde8945d868e7 in term 1.
I20260806 13:10:27.888677 8938 raft_consensus.cc:2454] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cf9a6d579ac4a84b84dde8945d868e7 in term 1.
I20260806 13:10:27.888710 8618 leader_election.cc:302] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a984459dc7544ce8cbda3ca1ba204aa, 6cf9a6d579ac4a84b84dde8945d868e7; no voters:
I20260806 13:10:27.888795 9038 raft_consensus.cc:2781] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:27.888947 9038 raft_consensus.cc:686] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cf9a6d579ac4a84b84dde8945d868e7, State: Running, Role: LEADER
I20260806 13:10:27.889034 9038 consensus_queue.cc:237] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a984459dc7544ce8cbda3ca1ba204aa" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40313 } } peers { permanent_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39887 } } peers { permanent_uuid: "25bc640efa60424099cc8be5198f2b54" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 32883 } }
I20260806 13:10:27.889832 8288 catalog_manager.cc:5495] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cf9a6d579ac4a84b84dde8945d868e7 (127.13.224.66). New cstate: current_term: 1 leader_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a984459dc7544ce8cbda3ca1ba204aa" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cf9a6d579ac4a84b84dde8945d868e7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25bc640efa60424099cc8be5198f2b54" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 32883 } health_report { overall_health: UNKNOWN } } }
I20260806 13:10:27.958416 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.956382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46752 (local address 127.13.224.66:39887)
0806 13:10:27.956422 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:27.956424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:27.956433 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:10:27.956498 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:27.956498 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:27.956514 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:27.956562 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:27.956668 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.957112 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.957810 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:27.957979 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:27.958037 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:27.958166 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:27.958252 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:27.958320 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:27.958352 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 13:10:27.958361 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:10:27.960086 8512 raft_consensus.cc:1261] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa [term 1 FOLLOWER]: Refusing update from remote peer 6cf9a6d579ac4a84b84dde8945d868e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:27.960086 8938 raft_consensus.cc:1261] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54 [term 1 FOLLOWER]: Refusing update from remote peer 6cf9a6d579ac4a84b84dde8945d868e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:27.960438 9044 consensus_queue.cc:1035] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a984459dc7544ce8cbda3ca1ba204aa" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:27.960541 9038 consensus_queue.cc:1035] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25bc640efa60424099cc8be5198f2b54" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 32883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:27.962889 9051 mvcc.cc:205] Tried to move back new op lower bound from 7315545407322009600 to 7315545407034093568. Current Snapshot: MvccSnapshot[applied={T|T < 7315545407322009600}]
I20260806 13:10:27.963934 9053 mvcc.cc:205] Tried to move back new op lower bound from 7315545407322009600 to 7315545407034093568. Current Snapshot: MvccSnapshot[applied={T|T < 7315545407322009600}]
I20260806 13:10:27.965657 9052 mvcc.cc:205] Tried to move back new op lower bound from 7315545407322009600 to 7315545407034093568. Current Snapshot: MvccSnapshot[applied={T|T < 7315545407322009600 or (T in {7315545407322009600})}]
I20260806 13:10:27.967368 8287 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:10:27.967448 8287 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:10:27.968142 8287 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89781d41d3e1447f8f8efe445ed8b22b: Sending DeleteTablet for 3 replicas of tablet 5b59d3ba62c54433b7f708d09c6524ff
I20260806 13:10:27.968461 8478 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b59d3ba62c54433b7f708d09c6524ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:27 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:40128
I20260806 13:10:27.968706 9063 tablet_replica.cc:330] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa: stopping tablet replica
I20260806 13:10:27.968799 9063 raft_consensus.cc:2229] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:10:27.968847 8696 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b59d3ba62c54433b7f708d09c6524ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:27 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:52524
I20260806 13:10:27.968907 9063 raft_consensus.cc:2258] T 5b59d3ba62c54433b7f708d09c6524ff P 6a984459dc7544ce8cbda3ca1ba204aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:27.969053 9064 tablet_replica.cc:330] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7: stopping tablet replica
I20260806 13:10:27.969055 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 8383
I20260806 13:10:27.969183 9064 raft_consensus.cc:2229] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:10:27.969298 9064 raft_consensus.cc:2258] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:27.969861 9064 ts_tablet_manager.cc:1905] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:10:27.971810 9064 ts_tablet_manager.cc:1918] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:10:27.971930 9064 log.cc:1196] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/5b59d3ba62c54433b7f708d09c6524ff
I20260806 13:10:27.972270 9064 ts_tablet_manager.cc:1939] T 5b59d3ba62c54433b7f708d09c6524ff P 6cf9a6d579ac4a84b84dde8945d868e7: Deleting consensus metadata
I20260806 13:10:27.972774 8264 catalog_manager.cc:4841] TS 6cf9a6d579ac4a84b84dde8945d868e7 (127.13.224.66:39887): tablet 5b59d3ba62c54433b7f708d09c6524ff (table test-table [id=94721309fd2f47ffafe18c171d6b5c61]) successfully deleted
I20260806 13:10:27.975895 8909 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b59d3ba62c54433b7f708d09c6524ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:27 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:59898
I20260806 13:10:27.976809 9065 tablet_replica.cc:330] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54: stopping tablet replica
I20260806 13:10:27.976909 9065 raft_consensus.cc:2229] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:10:27.977010 9065 raft_consensus.cc:2258] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:27.977406 9065 ts_tablet_manager.cc:1905] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:10:27.978744 9065 ts_tablet_manager.cc:1918] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:10:27.978809 9065 log.cc:1196] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/5b59d3ba62c54433b7f708d09c6524ff
I20260806 13:10:27.979058 9065 ts_tablet_manager.cc:1939] T 5b59d3ba62c54433b7f708d09c6524ff P 25bc640efa60424099cc8be5198f2b54: Deleting consensus metadata
I20260806 13:10:27.979413 8262 catalog_manager.cc:4841] TS 25bc640efa60424099cc8be5198f2b54 (127.13.224.67:32883): tablet 5b59d3ba62c54433b7f708d09c6524ff (table test-table [id=94721309fd2f47ffafe18c171d6b5c61]) successfully deleted
W20260806 13:10:27.981889 8264 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:40313 (error 108)
I20260806 13:10:27.982111 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 8593
I20260806 13:10:27.986289 9015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:27.982274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:40313 (local address 127.0.0.1:47814)
0806 13:10:27.986218 (+ 3944us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:27.986260 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:40313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3924}
W20260806 13:10:27.991967 8264 catalog_manager.cc:4584] TS 6a984459dc7544ce8cbda3ca1ba204aa (127.13.224.65:40313): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b59d3ba62c54433b7f708d09c6524ff: Network error: Client connection negotiation failed: client connection to 127.13.224.65:40313: connect: Connection refused (error 111)
I20260806 13:10:27.995644 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 8811
I20260806 13:10:28.004631 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 8240
2026-08-06T13:10:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2276 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T13:10:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:28Z Disabled control of system clock
I20260806 13:10:28.033162 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:41431
--webserver_interface=127.13.224.126
--webserver_port=0
--builtin_ntp_servers=127.13.224.84:46083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:41431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:10:28.110798 9070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:28.110980 9070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:28.110998 9070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:28.112087 9070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:28.112131 9070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:28.112144 9070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:28.112169 9070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:28.112190 9070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:28.113487 9070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:41431
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:41431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:28.113703 9070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:28.113950 9070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 13:10:28.118649 9070 server_base.cc:1034] running on GCE node
W20260806 13:10:28.118607 9076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:28.118621 9078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:28.118611 9080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:28.119029 9070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:28.119297 9070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:28.120435 9070 hybrid_clock.cc:648] HybridClock initialized: now 1786021828120423 us; error 28 us; skew 500 ppm
I20260806 13:10:28.128062 9070 webserver.cc:460] Webserver started at http://127.13.224.126:45891/ using document root <none> and password file <none>
I20260806 13:10:28.128289 9070 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:28.128338 9070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:28.128460 9070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:28.129407 9070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "a7daf80810814d99b873bfdc10648ec2"
format_stamp: "Formatted at 2026-08-06 13:10:28 on dist-test-slave-q9h9"
server_key: "d54fc91f4383f36b1b6527fb39c26df5"
server_key_iv: "9bb34ae53e0ed70eddaa0212e624800f"
server_key_version: "encryptionkey@0"
I20260806 13:10:28.129711 9070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "a7daf80810814d99b873bfdc10648ec2"
format_stamp: "Formatted at 2026-08-06 13:10:28 on dist-test-slave-q9h9"
server_key: "d54fc91f4383f36b1b6527fb39c26df5"
server_key_iv: "9bb34ae53e0ed70eddaa0212e624800f"
server_key_version: "encryptionkey@0"
I20260806 13:10:28.130899 9070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:10:28.131557 9088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:28.131708 9070 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:28.131799 9070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "a7daf80810814d99b873bfdc10648ec2"
format_stamp: "Formatted at 2026-08-06 13:10:28 on dist-test-slave-q9h9"
server_key: "d54fc91f4383f36b1b6527fb39c26df5"
server_key_iv: "9bb34ae53e0ed70eddaa0212e624800f"
server_key_version: "encryptionkey@0"
I20260806 13:10:28.131860 9070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:28.161774 9070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:28.162075 9070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:28.162207 9070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:28.165763 9070 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:41431
I20260806 13:10:28.165799 9162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:41431 every 8 connection(s)
I20260806 13:10:28.166105 9070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:28.167027 9163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:28.167140 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 9070
I20260806 13:10:28.167232 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:28.167402 14209 external_mini_cluster.cc:1428] Setting key ff65e33569a9d941314f0dd113e847df
I20260806 13:10:28.170277 9163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2: Bootstrap starting.
I20260806 13:10:28.170849 9163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:28.171101 9163 log.cc:824] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:28.171733 9163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2: No bootstrap required, opened a new log
I20260806 13:10:28.172691 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.167867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33594 (local address 127.13.224.126:41431)
0806 13:10:28.168068 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:28.168077 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:28.168122 (+ 45us) server_negotiation.cc:407] Connection header received
0806 13:10:28.168544 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:28.168554 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:28.168632 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:28.168774 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:28.168924 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.170237 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.170979 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.171395 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.171500 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.172140 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:28.172145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:28.172145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:28.172300 (+ 155us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:28.172330 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:28.172341 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:28.172394 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:28.172466 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 13:10:28.172540 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260806 13:10:28.173532 9163 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7daf80810814d99b873bfdc10648ec2" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41431 } }
I20260806 13:10:28.173666 9163 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:28.173698 9163 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7daf80810814d99b873bfdc10648ec2, State: Initialized, Role: FOLLOWER
I20260806 13:10:28.173806 9163 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7daf80810814d99b873bfdc10648ec2" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41431 } }
I20260806 13:10:28.173863 9163 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:28.173893 9163 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:28.173938 9163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:28.174449 9163 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7daf80810814d99b873bfdc10648ec2" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41431 } }
I20260806 13:10:28.174540 9163 leader_election.cc:302] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7daf80810814d99b873bfdc10648ec2; no voters:
I20260806 13:10:28.174692 9163 leader_election.cc:288] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:28.174732 9174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:28.174886 9174 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [term 1 LEADER]: Becoming Leader. State: Replica: a7daf80810814d99b873bfdc10648ec2, State: Running, Role: LEADER
I20260806 13:10:28.174937 9163 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:28.174973 9174 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7daf80810814d99b873bfdc10648ec2" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41431 } }
I20260806 13:10:28.175303 9176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7daf80810814d99b873bfdc10648ec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7daf80810814d99b873bfdc10648ec2" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41431 } } }
I20260806 13:10:28.175396 9176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:28.175312 9177 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7daf80810814d99b873bfdc10648ec2. Latest consensus state: current_term: 1 leader_uuid: "a7daf80810814d99b873bfdc10648ec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7daf80810814d99b873bfdc10648ec2" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41431 } } }
I20260806 13:10:28.175530 9177 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:28.175868 9186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:28.176307 9186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:28.177621 9186 catalog_manager.cc:1347] Generated new cluster ID: eb68d2a6e3ec414389662451b46212de
I20260806 13:10:28.177663 9186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:28.183212 9186 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:28.183676 9186 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:28.191926 9186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2: Generated new TSK 0
I20260806 13:10:28.192066 9186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:28.193993 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:41431
--builtin_ntp_servers=127.13.224.84:46083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:10:28.275099 9202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:28.275300 9202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:28.275331 9202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:28.276484 9202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:28.276522 9202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:28.276556 9202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:10:28.277873 9202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:41431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:28.278123 9202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:28.278398 9202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:28.283867 9216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:28.283875 9211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:28.283918 9212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:28.284099 9202 server_base.cc:1034] running on GCE node
I20260806 13:10:28.284320 9202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:28.284534 9202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:28.285671 9202 hybrid_clock.cc:648] HybridClock initialized: now 1786021828285660 us; error 30 us; skew 500 ppm
I20260806 13:10:28.286868 9202 webserver.cc:460] Webserver started at http://127.13.224.65:44203/ using document root <none> and password file <none>
I20260806 13:10:28.287106 9202 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:28.287153 9202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:28.287281 9202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:28.288236 9202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "d14ee3c07e224367813d65556d994a73"
format_stamp: "Formatted at 2026-08-06 13:10:28 on dist-test-slave-q9h9"
server_key: "236244d63d4b0a020c80c32c71e12257"
server_key_iv: "8adbed69e4737c800119590dab9234f3"
server_key_version: "encryptionkey@0"
I20260806 13:10:28.288563 9202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "d14ee3c07e224367813d65556d994a73"
format_stamp: "Formatted at 2026-08-06 13:10:28 on dist-test-slave-q9h9"
server_key: "236244d63d4b0a020c80c32c71e12257"
server_key_iv: "8adbed69e4737c800119590dab9234f3"
server_key_version: "encryptionkey@0"
I20260806 13:10:28.289742 9202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:10:28.290472 9224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:28.290632 9202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:28.290715 9202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "d14ee3c07e224367813d65556d994a73"
format_stamp: "Formatted at 2026-08-06 13:10:28 on dist-test-slave-q9h9"
server_key: "236244d63d4b0a020c80c32c71e12257"
server_key_iv: "8adbed69e4737c800119590dab9234f3"
server_key_version: "encryptionkey@0"
I20260806 13:10:28.290782 9202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:28.313035 9202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:28.313282 9202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:28.313405 9202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:28.313591 9202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:28.313874 9202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:28.313906 9202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:28.313932 9202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:28.313958 9202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:28.319470 9202 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:37065
I20260806 13:10:28.319509 9393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:37065 every 8 connection(s)
I20260806 13:10:28.319784 9202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:10:28.320997 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 9202
I20260806 13:10:28.321094 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:10:28.321297 14209 external_mini_cluster.cc:1428] Setting key 09486efc1761202826aae9065bcb087d
I20260806 13:10:28.322216 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:41431
--builtin_ntp_servers=127.13.224.84:46083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 13:10:28.325217 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.320411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:42029 (local address 127.13.224.126:41431)
0806 13:10:28.320457 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:28.320460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:28.320682 (+ 222us) server_negotiation.cc:407] Connection header received
0806 13:10:28.321265 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:28.321266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:28.321278 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:28.321322 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:28.322110 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.322647 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.323930 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.324195 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.324262 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.324943 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:28.324945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:28.324945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:28.324950 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:28.324966 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:28.324970 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:28.325024 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:28.325118 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 13:10:28.325144 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:10:28.325361 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.320404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:41431 (local address 127.13.224.65:42029)
0806 13:10:28.320576 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:28.320608 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:28.321160 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:28.321407 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:28.321411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:28.321422 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:28.321999 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.322005 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.322737 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.322738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.323843 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.323846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.324262 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.324263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.324365 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.324823 (+ 458us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:10:28.324823 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:28.324848 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:10:28.324868 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:28.325043 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:28.325048 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:28.325051 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:10:28.325113 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 13:10:28.325188 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260806 13:10:28.325786 9394 heartbeater.cc:347] Connected to a master server at 127.13.224.126:41431
I20260806 13:10:28.325865 9394 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:28.326048 9394 heartbeater.cc:511] Master 127.13.224.126:41431 requested a full tablet report, sending...
I20260806 13:10:28.326542 9113 ts_manager.cc:194] Registered new tserver with Master: d14ee3c07e224367813d65556d994a73 (127.13.224.65:37065)
I20260806 13:10:28.327140 9113 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.224.65:42029
W20260806 13:10:28.401780 9400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:28.401955 9400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:28.401985 9400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:28.403090 9400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:28.403127 9400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:28.403160 9400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:10:28.404451 9400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:41431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:28.404666 9400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:28.404935 9400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:28.409456 9408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:28.409504 9409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:28.409475 9411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:28.409824 9400 server_base.cc:1034] running on GCE node
I20260806 13:10:28.410004 9400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:28.410233 9400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:28.411360 9400 hybrid_clock.cc:648] HybridClock initialized: now 1786021828411349 us; error 28 us; skew 500 ppm
I20260806 13:10:28.412429 9400 webserver.cc:460] Webserver started at http://127.13.224.66:40769/ using document root <none> and password file <none>
I20260806 13:10:28.412626 9400 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:28.412667 9400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:28.412813 9400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:28.413767 9400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "483b44656daa419cafd7e15d703fd117"
format_stamp: "Formatted at 2026-08-06 13:10:28 on dist-test-slave-q9h9"
server_key: "afcc003d9cf3a18d7dd8b6c38da62775"
server_key_iv: "2e055c8548266a6c29f3dafba4620c25"
server_key_version: "encryptionkey@0"
I20260806 13:10:28.414119 9400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "483b44656daa419cafd7e15d703fd117"
format_stamp: "Formatted at 2026-08-06 13:10:28 on dist-test-slave-q9h9"
server_key: "afcc003d9cf3a18d7dd8b6c38da62775"
server_key_iv: "2e055c8548266a6c29f3dafba4620c25"
server_key_version: "encryptionkey@0"
I20260806 13:10:28.415274 9400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:10:28.415884 9417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:28.416028 9400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:10:28.416105 9400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "483b44656daa419cafd7e15d703fd117"
format_stamp: "Formatted at 2026-08-06 13:10:28 on dist-test-slave-q9h9"
server_key: "afcc003d9cf3a18d7dd8b6c38da62775"
server_key_iv: "2e055c8548266a6c29f3dafba4620c25"
server_key_version: "encryptionkey@0"
I20260806 13:10:28.416167 9400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:28.426533 9400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:28.426749 9400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:28.426859 9400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:28.427026 9400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:28.427289 9400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:28.427321 9400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:28.427345 9400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:28.427372 9400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:28.432477 9400 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:43065
I20260806 13:10:28.432539 9591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:43065 every 8 connection(s)
I20260806 13:10:28.432780 9400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:10:28.437497 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.433365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:43657 (local address 127.13.224.126:41431)
0806 13:10:28.433406 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:28.433408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:28.433590 (+ 182us) server_negotiation.cc:407] Connection header received
0806 13:10:28.434058 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:28.434059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:28.434072 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:28.434122 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:28.434808 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.435290 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.436241 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.436475 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.436533 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.437253 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:28.437255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:28.437255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:28.437260 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:28.437275 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:28.437280 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:28.437325 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:28.437395 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 13:10:28.437420 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:10:28.437670 9597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.433344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:41431 (local address 127.13.224.66:43657)
0806 13:10:28.433493 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:28.433521 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:28.433934 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:28.434182 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:28.434186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:28.434195 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:28.434713 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.434718 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.435375 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.435376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.436158 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.436160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.436551 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.436552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.436681 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.437119 (+ 438us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:10:28.437119 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:28.437146 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:10:28.437166 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:28.437330 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:28.437336 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:28.437338 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:10:28.437397 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 13:10:28.437487 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
I20260806 13:10:28.437969 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 9400
I20260806 13:10:28.438086 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:10:28.438083 9593 heartbeater.cc:347] Connected to a master server at 127.13.224.126:41431
I20260806 13:10:28.438159 9593 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:28.438273 14209 external_mini_cluster.cc:1428] Setting key 85e62a17b6d98ba757f29ce9a78c0d5f
I20260806 13:10:28.438320 9593 heartbeater.cc:511] Master 127.13.224.126:41431 requested a full tablet report, sending...
I20260806 13:10:28.438666 9111 ts_manager.cc:194] Registered new tserver with Master: 483b44656daa419cafd7e15d703fd117 (127.13.224.66:43065)
I20260806 13:10:28.439086 9111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.224.66:43657
I20260806 13:10:28.439193 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:41431
--builtin_ntp_servers=127.13.224.84:46083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:10:28.518275 9599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:28.518492 9599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:28.518530 9599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:28.519652 9599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:28.519687 9599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:28.519711 9599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:10:28.520995 9599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:41431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:28.521178 9599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:28.521404 9599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:28.525982 9608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:28.526079 9599 server_base.cc:1034] running on GCE node
W20260806 13:10:28.525962 9612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:28.525964 9607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:28.526319 9599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:28.526548 9599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:28.527688 9599 hybrid_clock.cc:648] HybridClock initialized: now 1786021828527667 us; error 36 us; skew 500 ppm
I20260806 13:10:28.528788 9599 webserver.cc:460] Webserver started at http://127.13.224.67:41769/ using document root <none> and password file <none>
I20260806 13:10:28.528991 9599 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:28.529034 9599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:28.529156 9599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:28.530184 9599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "2d1a01b57c154c588e36a91fba2ff4b6"
format_stamp: "Formatted at 2026-08-06 13:10:28 on dist-test-slave-q9h9"
server_key: "06570a7202b055fde36d7bed98b7ec8b"
server_key_iv: "2b59052676408883f04ae6f3e6a4eba7"
server_key_version: "encryptionkey@0"
I20260806 13:10:28.530490 9599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "2d1a01b57c154c588e36a91fba2ff4b6"
format_stamp: "Formatted at 2026-08-06 13:10:28 on dist-test-slave-q9h9"
server_key: "06570a7202b055fde36d7bed98b7ec8b"
server_key_iv: "2b59052676408883f04ae6f3e6a4eba7"
server_key_version: "encryptionkey@0"
I20260806 13:10:28.531632 9599 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:10:28.532219 9618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:28.532356 9599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:28.532434 9599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "2d1a01b57c154c588e36a91fba2ff4b6"
format_stamp: "Formatted at 2026-08-06 13:10:28 on dist-test-slave-q9h9"
server_key: "06570a7202b055fde36d7bed98b7ec8b"
server_key_iv: "2b59052676408883f04ae6f3e6a4eba7"
server_key_version: "encryptionkey@0"
I20260806 13:10:28.532492 9599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:28.546713 9599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:28.546937 9599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:28.547049 9599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:28.547228 9599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:28.547498 9599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:28.547530 9599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:28.547555 9599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:28.547587 9599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:28.552886 9599 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:42607
I20260806 13:10:28.552947 9793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:42607 every 8 connection(s)
I20260806 13:10:28.553191 9599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:10:28.554100 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 9599
I20260806 13:10:28.554200 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:10:28.554363 14209 external_mini_cluster.cc:1428] Setting key 2c7d2058289a7fd7c94751c7b29dc6a1
I20260806 13:10:28.557842 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.553800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:45695 (local address 127.13.224.126:41431)
0806 13:10:28.553844 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:28.553846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:28.554061 (+ 215us) server_negotiation.cc:407] Connection header received
0806 13:10:28.554575 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:28.554577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:28.554589 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:28.554635 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:28.555274 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.555748 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.556654 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.556853 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.556907 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.557600 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:28.557602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:28.557602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:28.557606 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:28.557621 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:28.557625 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:28.557671 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:28.557739 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 13:10:28.557764 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:10:28.558048 9798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.553757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:41431 (local address 127.13.224.67:45695)
0806 13:10:28.553932 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:28.553967 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:28.554484 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:28.554712 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:28.554716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:28.554727 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:28.555185 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.555189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.555832 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.555833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.556577 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.556579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.556924 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.556925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.557051 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.557478 (+ 427us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:10:28.557479 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:28.557504 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:10:28.557524 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:28.557677 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:28.557682 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:28.557684 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:10:28.557742 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 13:10:28.557841 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260806 13:10:28.558423 9794 heartbeater.cc:347] Connected to a master server at 127.13.224.126:41431
I20260806 13:10:28.558498 9794 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:28.558626 9794 heartbeater.cc:511] Master 127.13.224.126:41431 requested a full tablet report, sending...
I20260806 13:10:28.558910 9111 ts_manager.cc:194] Registered new tserver with Master: 2d1a01b57c154c588e36a91fba2ff4b6 (127.13.224.67:42607)
I20260806 13:10:28.559257 9111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.224.67:45695
I20260806 13:10:28.565714 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:10:28.569221 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.566970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33610 (local address 127.13.224.126:41431)
0806 13:10:28.567025 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:28.567027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:28.567099 (+ 72us) server_negotiation.cc:407] Connection header received
0806 13:10:28.567138 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:28.567139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:28.567146 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:28.567184 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:28.567308 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.567728 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.568390 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.568549 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.568586 (+ 37us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.569030 (+ 444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:28.569031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:28.569032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:28.569035 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:28.569045 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:28.569048 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:28.569108 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:28.569147 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 13:10:28.569155 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 13:10:28.570668 9111 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:10:28.572088 9111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:10:28.577796 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.573978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:37065 (local address 127.0.0.1:59660)
0806 13:10:28.574197 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:28.574235 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:28.574294 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:28.574575 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:28.574577 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:28.574579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:28.574637 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.574639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.575627 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.575628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.576579 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.576583 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.577501 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.577503 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.577579 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.577591 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 13:10:28.577668 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 13:10:28.577718 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":35,"thread_start_us":68,"threads_started":1}
I20260806 13:10:28.577800 9812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.574161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.13.224.65:37065)
0806 13:10:28.574441 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:28.574445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:28.574454 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:10:28.574491 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:28.574492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:28.574507 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:28.574572 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:28.574710 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.575552 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.576713 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.577420 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.577490 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.577569 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:28.577684 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 13:10:28.577721 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":90,"threads_started":1}
I20260806 13:10:28.578001 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.573978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:43065 (local address 127.0.0.1:52126)
0806 13:10:28.574341 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:28.574351 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:28.574397 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:28.574986 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:28.574987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:28.574988 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:28.575022 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.575023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.575973 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.575974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.577020 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.577023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.577799 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.577800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.577855 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.577859 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:10:28.577910 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 13:10:28.577943 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260806 13:10:28.578301 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.574176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52126 (local address 127.13.224.66:43065)
0806 13:10:28.574757 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:28.574761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:28.574771 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:10:28.574819 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:28.574851 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:28.574868 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:28.574967 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:28.575080 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.575871 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.577094 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.577625 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.577682 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.577764 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:28.578168 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 13:10:28.578210 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":88,"threads_started":1}
I20260806 13:10:28.578461 9290 tablet_service.cc:1467] Processing CreateTablet for tablet 23ee6646f1a94800a66ce9b3ce96b1ff (DEFAULT_TABLE table=test-table [id=3b384e7239074f5ba8182919f67860dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:28.578826 9290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23ee6646f1a94800a66ce9b3ce96b1ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:28.578855 9487 tablet_service.cc:1467] Processing CreateTablet for tablet 23ee6646f1a94800a66ce9b3ce96b1ff (DEFAULT_TABLE table=test-table [id=3b384e7239074f5ba8182919f67860dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:28.579114 9809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.573993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:42607 (local address 127.0.0.1:43566)
0806 13:10:28.574398 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:28.574406 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:28.574456 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:28.574590 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:28.574591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:28.574593 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:28.574648 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.574650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.576761 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.576762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.577770 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.577773 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.578891 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.578892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.578951 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.578955 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:10:28.579005 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 13:10:28.579040 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":113,"threads_started":1}
I20260806 13:10:28.579186 9487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23ee6646f1a94800a66ce9b3ce96b1ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:28.579447 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.574198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43566 (local address 127.13.224.67:42607)
0806 13:10:28.574358 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:28.574363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:28.574470 (+ 107us) server_negotiation.cc:407] Connection header received
0806 13:10:28.574508 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:28.574509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:28.574520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:28.574584 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:28.575627 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.576639 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.577975 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.578716 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.578780 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.578852 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:28.579256 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 13:10:28.579292 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260806 13:10:28.581374 9815 tablet_bootstrap.cc:492] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73: Bootstrap starting.
I20260806 13:10:28.581873 9691 tablet_service.cc:1467] Processing CreateTablet for tablet 23ee6646f1a94800a66ce9b3ce96b1ff (DEFAULT_TABLE table=test-table [id=3b384e7239074f5ba8182919f67860dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:28.582002 9815 tablet_bootstrap.cc:654] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:28.582201 9691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23ee6646f1a94800a66ce9b3ce96b1ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:28.582355 9815 log.cc:824] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:28.583073 9815 tablet_bootstrap.cc:492] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73: No bootstrap required, opened a new log
I20260806 13:10:28.583160 9815 ts_tablet_manager.cc:1397] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 13:10:28.585160 9818 tablet_bootstrap.cc:492] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6: Bootstrap starting.
I20260806 13:10:28.585263 9816 tablet_bootstrap.cc:492] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117: Bootstrap starting.
I20260806 13:10:28.585546 9815 raft_consensus.cc:348] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "483b44656daa419cafd7e15d703fd117" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43065 } } peers { permanent_uuid: "d14ee3c07e224367813d65556d994a73" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 37065 } } peers { permanent_uuid: "2d1a01b57c154c588e36a91fba2ff4b6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42607 } }
I20260806 13:10:28.585703 9815 raft_consensus.cc:374] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:28.585744 9815 raft_consensus.cc:729] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14ee3c07e224367813d65556d994a73, State: Initialized, Role: FOLLOWER
I20260806 13:10:28.585839 9818 tablet_bootstrap.cc:654] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:28.585896 9816 tablet_bootstrap.cc:654] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:28.585868 9815 consensus_queue.cc:260] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "483b44656daa419cafd7e15d703fd117" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43065 } } peers { permanent_uuid: "d14ee3c07e224367813d65556d994a73" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 37065 } } peers { permanent_uuid: "2d1a01b57c154c588e36a91fba2ff4b6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42607 } }
I20260806 13:10:28.586155 9815 ts_tablet_manager.cc:1428] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 13:10:28.586187 9394 heartbeater.cc:503] Master 127.13.224.126:41431 was elected leader, sending a full tablet report...
I20260806 13:10:28.586202 9818 log.cc:824] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:28.586472 9816 log.cc:824] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:28.586894 9818 tablet_bootstrap.cc:492] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6: No bootstrap required, opened a new log
I20260806 13:10:28.586961 9818 ts_tablet_manager.cc:1397] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 13:10:28.587267 9816 tablet_bootstrap.cc:492] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117: No bootstrap required, opened a new log
I20260806 13:10:28.587330 9816 ts_tablet_manager.cc:1397] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 13:10:28.588738 9818 raft_consensus.cc:348] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "483b44656daa419cafd7e15d703fd117" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43065 } } peers { permanent_uuid: "d14ee3c07e224367813d65556d994a73" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 37065 } } peers { permanent_uuid: "2d1a01b57c154c588e36a91fba2ff4b6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42607 } }
I20260806 13:10:28.588862 9818 raft_consensus.cc:374] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:28.588892 9818 raft_consensus.cc:729] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1a01b57c154c588e36a91fba2ff4b6, State: Initialized, Role: FOLLOWER
I20260806 13:10:28.588994 9818 consensus_queue.cc:260] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "483b44656daa419cafd7e15d703fd117" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43065 } } peers { permanent_uuid: "d14ee3c07e224367813d65556d994a73" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 37065 } } peers { permanent_uuid: "2d1a01b57c154c588e36a91fba2ff4b6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42607 } }
I20260806 13:10:28.589033 9816 raft_consensus.cc:348] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "483b44656daa419cafd7e15d703fd117" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43065 } } peers { permanent_uuid: "d14ee3c07e224367813d65556d994a73" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 37065 } } peers { permanent_uuid: "2d1a01b57c154c588e36a91fba2ff4b6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42607 } }
I20260806 13:10:28.589179 9816 raft_consensus.cc:374] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:28.589224 9816 raft_consensus.cc:729] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 483b44656daa419cafd7e15d703fd117, State: Initialized, Role: FOLLOWER
I20260806 13:10:28.589236 9818 ts_tablet_manager.cc:1428] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 13:10:28.589267 9794 heartbeater.cc:503] Master 127.13.224.126:41431 was elected leader, sending a full tablet report...
I20260806 13:10:28.589330 9816 consensus_queue.cc:260] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "483b44656daa419cafd7e15d703fd117" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43065 } } peers { permanent_uuid: "d14ee3c07e224367813d65556d994a73" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 37065 } } peers { permanent_uuid: "2d1a01b57c154c588e36a91fba2ff4b6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42607 } }
I20260806 13:10:28.589566 9816 ts_tablet_manager.cc:1428] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 13:10:28.589680 9593 heartbeater.cc:503] Master 127.13.224.126:41431 was elected leader, sending a full tablet report...
W20260806 13:10:28.683252 9594 tablet.cc:2369] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:10:28.701784 9828 raft_consensus.cc:482] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:10:28.701885 9828 raft_consensus.cc:504] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "483b44656daa419cafd7e15d703fd117" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43065 } } peers { permanent_uuid: "d14ee3c07e224367813d65556d994a73" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 37065 } } peers { permanent_uuid: "2d1a01b57c154c588e36a91fba2ff4b6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42607 } }
I20260806 13:10:28.702258 9828 leader_election.cc:288] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d14ee3c07e224367813d65556d994a73 (127.13.224.65:37065), 2d1a01b57c154c588e36a91fba2ff4b6 (127.13.224.67:42607)
I20260806 13:10:28.704962 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.702533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:54991 (local address 127.13.224.67:42607)
0806 13:10:28.702577 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:28.702579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:28.702588 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:10:28.702619 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:28.702620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:28.702629 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:28.702688 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:28.702862 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.703359 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.704286 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.704644 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.704706 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.704715 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:28.704855 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 13:10:28.704895 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:10:28.704965 9812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.702469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:56007 (local address 127.13.224.65:37065)
0806 13:10:28.702518 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:28.702520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:28.702689 (+ 169us) server_negotiation.cc:407] Connection header received
0806 13:10:28.702752 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:28.702753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:28.702765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:28.702815 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:28.702959 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.703433 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.704309 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.704655 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.704720 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.704728 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:28.704865 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 13:10:28.704901 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:10:28.704965 9597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.702418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:42607 (local address 127.13.224.66:54991)
0806 13:10:28.702468 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:28.702480 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:28.702525 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:28.702719 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:28.702720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:28.702722 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:28.702794 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.702797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.703430 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.703431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.704197 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.704200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.704725 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.704726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.704791 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.704794 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:10:28.704834 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 13:10:28.704880 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 13:10:28.704981 9829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.702424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:37065 (local address 127.13.224.66:56007)
0806 13:10:28.702619 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:28.702629 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:28.702688 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:28.702829 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:28.702830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:28.702831 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:28.702902 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.702904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.703507 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.703508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.704223 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:28.704225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.704730 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.704731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:28.704802 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.704805 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:10:28.704855 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 13:10:28.704892 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":41,"thread_start_us":88,"threads_started":1}
I20260806 13:10:28.705346 9721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ee6646f1a94800a66ce9b3ce96b1ff" candidate_uuid: "483b44656daa419cafd7e15d703fd117" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1a01b57c154c588e36a91fba2ff4b6" is_pre_election: true
I20260806 13:10:28.705467 9326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ee6646f1a94800a66ce9b3ce96b1ff" candidate_uuid: "483b44656daa419cafd7e15d703fd117" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14ee3c07e224367813d65556d994a73" is_pre_election: true
I20260806 13:10:28.705509 9721 raft_consensus.cc:2454] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 483b44656daa419cafd7e15d703fd117 in term 0.
I20260806 13:10:28.705596 9326 raft_consensus.cc:2454] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 483b44656daa419cafd7e15d703fd117 in term 0.
I20260806 13:10:28.705736 9421 leader_election.cc:302] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1a01b57c154c588e36a91fba2ff4b6, 483b44656daa419cafd7e15d703fd117; no voters:
I20260806 13:10:28.705914 9828 raft_consensus.cc:2781] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:10:28.705963 9828 raft_consensus.cc:482] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:10:28.705978 9828 raft_consensus.cc:3037] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:28.706696 9828 raft_consensus.cc:504] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "483b44656daa419cafd7e15d703fd117" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43065 } } peers { permanent_uuid: "d14ee3c07e224367813d65556d994a73" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 37065 } } peers { permanent_uuid: "2d1a01b57c154c588e36a91fba2ff4b6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42607 } }
I20260806 13:10:28.706815 9828 leader_election.cc:288] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [CANDIDATE]: Term 1 election: Requested vote from peers d14ee3c07e224367813d65556d994a73 (127.13.224.65:37065), 2d1a01b57c154c588e36a91fba2ff4b6 (127.13.224.67:42607)
I20260806 13:10:28.706960 9721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ee6646f1a94800a66ce9b3ce96b1ff" candidate_uuid: "483b44656daa419cafd7e15d703fd117" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1a01b57c154c588e36a91fba2ff4b6"
I20260806 13:10:28.706997 9326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ee6646f1a94800a66ce9b3ce96b1ff" candidate_uuid: "483b44656daa419cafd7e15d703fd117" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14ee3c07e224367813d65556d994a73"
I20260806 13:10:28.707046 9721 raft_consensus.cc:3037] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:28.707057 9326 raft_consensus.cc:3037] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:28.707932 9326 raft_consensus.cc:2454] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 483b44656daa419cafd7e15d703fd117 in term 1.
I20260806 13:10:28.707932 9721 raft_consensus.cc:2454] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 483b44656daa419cafd7e15d703fd117 in term 1.
I20260806 13:10:28.708096 9421 leader_election.cc:302] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1a01b57c154c588e36a91fba2ff4b6, 483b44656daa419cafd7e15d703fd117; no voters:
I20260806 13:10:28.708175 9828 raft_consensus.cc:2781] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:28.708294 9828 raft_consensus.cc:686] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [term 1 LEADER]: Becoming Leader. State: Replica: 483b44656daa419cafd7e15d703fd117, State: Running, Role: LEADER
I20260806 13:10:28.708370 9828 consensus_queue.cc:237] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "483b44656daa419cafd7e15d703fd117" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43065 } } peers { permanent_uuid: "d14ee3c07e224367813d65556d994a73" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 37065 } } peers { permanent_uuid: "2d1a01b57c154c588e36a91fba2ff4b6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42607 } }
I20260806 13:10:28.709123 9113 catalog_manager.cc:5495] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 reported cstate change: term changed from 0 to 1, leader changed from <none> to 483b44656daa419cafd7e15d703fd117 (127.13.224.66). New cstate: current_term: 1 leader_uuid: "483b44656daa419cafd7e15d703fd117" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "483b44656daa419cafd7e15d703fd117" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 43065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d14ee3c07e224367813d65556d994a73" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 37065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d1a01b57c154c588e36a91fba2ff4b6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42607 } health_report { overall_health: UNKNOWN } } }
I20260806 13:10:28.720254 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.718001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52134 (local address 127.13.224.66:43065)
0806 13:10:28.718053 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:28.718055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:28.718077 (+ 22us) server_negotiation.cc:407] Connection header received
0806 13:10:28.718177 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:28.718178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:28.718194 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:28.718251 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:28.718374 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.718832 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.719534 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:28.719726 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:28.719778 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:28.719941 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:28.720026 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:28.720106 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:28.720158 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 13:10:28.720166 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 13:10:28.721685 9721 raft_consensus.cc:1261] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6 [term 1 FOLLOWER]: Refusing update from remote peer 483b44656daa419cafd7e15d703fd117: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:28.721686 9326 raft_consensus.cc:1261] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73 [term 1 FOLLOWER]: Refusing update from remote peer 483b44656daa419cafd7e15d703fd117: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:28.721969 9832 consensus_queue.cc:1035] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d14ee3c07e224367813d65556d994a73" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 37065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:28.722096 9828 consensus_queue.cc:1035] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d1a01b57c154c588e36a91fba2ff4b6" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 42607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:28.726567 9111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:10:28.726641 9111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:10:28.727275 9111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7daf80810814d99b873bfdc10648ec2: Sending DeleteTablet for 3 replicas of tablet 23ee6646f1a94800a66ce9b3ce96b1ff
I20260806 13:10:28.727571 9487 tablet_service.cc:1514] Processing DeleteTablet for tablet 23ee6646f1a94800a66ce9b3ce96b1ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:28 UTC) from {username='slave'} at 127.0.0.1:52126
I20260806 13:10:28.727610 9290 tablet_service.cc:1514] Processing DeleteTablet for tablet 23ee6646f1a94800a66ce9b3ce96b1ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:28 UTC) from {username='slave'} at 127.0.0.1:59660
I20260806 13:10:28.727833 9847 tablet_replica.cc:330] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73: stopping tablet replica
I20260806 13:10:28.727974 9847 raft_consensus.cc:2229] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:10:28.727952 9691 tablet_service.cc:1514] Processing DeleteTablet for tablet 23ee6646f1a94800a66ce9b3ce96b1ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:28 UTC) from {username='slave'} at 127.0.0.1:43566
I20260806 13:10:28.728134 9847 raft_consensus.cc:2258] T 23ee6646f1a94800a66ce9b3ce96b1ff P d14ee3c07e224367813d65556d994a73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:28.728179 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 9202
I20260806 13:10:28.742383 9849 tablet_replica.cc:330] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6: stopping tablet replica
I20260806 13:10:28.742555 9846 tablet_replica.cc:330] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117: stopping tablet replica
I20260806 13:10:28.742709 9846 raft_consensus.cc:2229] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:10:28.742828 9849 raft_consensus.cc:2229] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:10:28.742954 9846 raft_consensus.cc:2258] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:28.743080 9849 raft_consensus.cc:2258] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:28.745462 9846 ts_tablet_manager.cc:1905] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:10:28.747068 9849 ts_tablet_manager.cc:1905] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:10:28.748183 9849 ts_tablet_manager.cc:1918] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:10:28.748251 9849 log.cc:1196] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/23ee6646f1a94800a66ce9b3ce96b1ff
I20260806 13:10:28.748458 9849 ts_tablet_manager.cc:1939] T 23ee6646f1a94800a66ce9b3ce96b1ff P 2d1a01b57c154c588e36a91fba2ff4b6: Deleting consensus metadata
I20260806 13:10:28.748823 9846 ts_tablet_manager.cc:1918] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:10:28.748901 9846 log.cc:1196] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/23ee6646f1a94800a66ce9b3ce96b1ff
I20260806 13:10:28.748875 9093 catalog_manager.cc:4841] TS 2d1a01b57c154c588e36a91fba2ff4b6 (127.13.224.67:42607): tablet 23ee6646f1a94800a66ce9b3ce96b1ff (table test-table [id=3b384e7239074f5ba8182919f67860dd]) successfully deleted
I20260806 13:10:28.749135 9846 ts_tablet_manager.cc:1939] T 23ee6646f1a94800a66ce9b3ce96b1ff P 483b44656daa419cafd7e15d703fd117: Deleting consensus metadata
I20260806 13:10:28.749513 9089 catalog_manager.cc:4841] TS 483b44656daa419cafd7e15d703fd117 (127.13.224.66:43065): tablet 23ee6646f1a94800a66ce9b3ce96b1ff (table test-table [id=3b384e7239074f5ba8182919f67860dd]) successfully deleted
W20260806 13:10:28.750428 9092 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:37065 (error 108)
I20260806 13:10:28.750865 9809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:28.750757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:37065 (local address 127.0.0.1:59676)
0806 13:10:28.750802 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:28.750837 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 13:10:28.750964 9092 catalog_manager.cc:4584] TS d14ee3c07e224367813d65556d994a73 (127.13.224.65:37065): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 23ee6646f1a94800a66ce9b3ce96b1ff: Network error: Client connection negotiation failed: client connection to 127.13.224.65:37065: connect: Connection refused (error 111)
I20260806 13:10:28.751269 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 9400
I20260806 13:10:28.761075 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 9599
I20260806 13:10:28.773957 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 9070
2026-08-06T13:10:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (774 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:28 dist-test-slave-q9h9 krb5kdc[9854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:28 dist-test-slave-q9h9 krb5kdc[9854](info): set up 2 sockets
Aug 06 13:10:28 dist-test-slave-q9h9 krb5kdc[9854](info): commencing operation
krb5kdc: starting...
W20260806 13:10:29.888567 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:29 dist-test-slave-q9h9 krb5kdc[9854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:10:29.995972 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:43195
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:43195
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:30.075606 9871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:30.075809 9871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:30.075829 9871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:30.076901 9871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:30.076933 9871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:30.076946 9871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:30.076957 9871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:30.076970 9871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:30.078406 9871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:43195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:43195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:30.078593 9871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:30.078760 9871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:30.083331 9883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:30.083323 9887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:30.083338 9871 server_base.cc:1034] running on GCE node
W20260806 13:10:30.083334 9882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:30.083736 9871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:30.084028 9871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:30.085201 9871 hybrid_clock.cc:648] HybridClock initialized: now 1786021830085182 us; error 29 us; skew 500 ppm
Aug 06 13:10:30 dist-test-slave-q9h9 krb5kdc[9854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021830, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:30.087616 9871 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:10:30.088032 9871 webserver.cc:460] Webserver started at http://127.13.224.126:34259/ using document root <none> and password file <none>
I20260806 13:10:30.088228 9871 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:30.088276 9871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:30.088398 9871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:30.089383 9871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "c349be70d0d344f99146ff66ced69e0f"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "cfbbf96cd512fba174dc440c67526116"
server_key_iv: "9a93e2e79fb8db9d3c78f8d27ec89d65"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.089705 9871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "c349be70d0d344f99146ff66ced69e0f"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "cfbbf96cd512fba174dc440c67526116"
server_key_iv: "9a93e2e79fb8db9d3c78f8d27ec89d65"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.090943 9871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:10:30.091570 9900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:30.091713 9871 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:10:30.091797 9871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "c349be70d0d344f99146ff66ced69e0f"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "cfbbf96cd512fba174dc440c67526116"
server_key_iv: "9a93e2e79fb8db9d3c78f8d27ec89d65"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.091854 9871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:30.113631 9871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:30.115700 9871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:30.115837 9871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:30.119598 9871 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:43195
I20260806 13:10:30.119634 9982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:43195 every 8 connection(s)
I20260806 13:10:30.119963 9871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:30.120772 9983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:30.123010 9983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f: Bootstrap starting.
I20260806 13:10:30.123569 9983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:30.123822 9983 log.cc:824] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:30.124478 9983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f: No bootstrap required, opened a new log
I20260806 13:10:30.125984 9983 raft_consensus.cc:348] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c349be70d0d344f99146ff66ced69e0f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43195 } }
I20260806 13:10:30.126145 9983 raft_consensus.cc:374] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:30.126168 9983 raft_consensus.cc:729] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c349be70d0d344f99146ff66ced69e0f, State: Initialized, Role: FOLLOWER
I20260806 13:10:30.126313 9983 consensus_queue.cc:260] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c349be70d0d344f99146ff66ced69e0f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43195 } }
I20260806 13:10:30.126377 9983 raft_consensus.cc:388] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:30.126398 9983 raft_consensus.cc:482] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:30.126425 9983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:30.126907 9983 raft_consensus.cc:504] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c349be70d0d344f99146ff66ced69e0f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43195 } }
I20260806 13:10:30.126992 9983 leader_election.cc:302] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c349be70d0d344f99146ff66ced69e0f; no voters:
I20260806 13:10:30.127149 9983 leader_election.cc:288] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:30.127208 9988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:30.127385 9988 raft_consensus.cc:686] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [term 1 LEADER]: Becoming Leader. State: Replica: c349be70d0d344f99146ff66ced69e0f, State: Running, Role: LEADER
I20260806 13:10:30.127403 9983 sys_catalog.cc:564] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:30.127480 9988 consensus_queue.cc:237] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c349be70d0d344f99146ff66ced69e0f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43195 } }
I20260806 13:10:30.127859 9992 sys_catalog.cc:455] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c349be70d0d344f99146ff66ced69e0f. Latest consensus state: current_term: 1 leader_uuid: "c349be70d0d344f99146ff66ced69e0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c349be70d0d344f99146ff66ced69e0f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43195 } } }
I20260806 13:10:30.127941 9992 sys_catalog.cc:458] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:30.127967 9991 sys_catalog.cc:455] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c349be70d0d344f99146ff66ced69e0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c349be70d0d344f99146ff66ced69e0f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43195 } } }
I20260806 13:10:30.128016 9991 sys_catalog.cc:458] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:30.128659 10000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:30.129151 10000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:30.130077 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 9871
I20260806 13:10:30.130187 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:30.130388 14209 external_mini_cluster.cc:1428] Setting key e591d346ff38d18b5ef66e264d784b3c
I20260806 13:10:30.130972 10000 catalog_manager.cc:1347] Generated new cluster ID: d12e994c98294cc1a3215c664f488775
I20260806 13:10:30.131021 10000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:10:30 dist-test-slave-q9h9 krb5kdc[9854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:30.141456 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.130816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41006 (local address 127.13.224.126:43195)
0806 13:10:30.131244 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:30.131253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:30.131286 (+ 33us) server_negotiation.cc:407] Connection header received
0806 13:10:30.131394 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:30.131401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:30.131456 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:30.131587 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:30.132251 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.133675 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.134676 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.135315 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.135430 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.138122 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:30.138127 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:30.138127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:30.138139 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:30.139954 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:30.140338 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:30.140341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:30.140341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:30.140432 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:30.140609 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:30.140609 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:30.140610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:30.140887 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:30.141014 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:30.141111 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 13:10:30.141231 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":70,"threads_started":1}
I20260806 13:10:30.150664 10000 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:30.151120 10000 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:30.159142 10000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f: Generated new TSK 0
I20260806 13:10:30.159305 10000 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.224.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
Entry for principal kudu/127.13.224.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab.
I20260806 13:10:30.190634 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43195
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:30.268996 10023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:30.269176 10023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:30.269197 10023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:30.270311 10023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:30.270350 10023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:30.270378 10023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:10:30.271770 10023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.65.keytab
--principal=kudu/127.13.224.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:30.271981 10023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:30.272178 10023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:30.276901 10032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:30.276889 10036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:30.276917 10033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:30.277125 10023 server_base.cc:1034] running on GCE node
I20260806 13:10:30.277304 10023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:30.277489 10023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:30.278633 10023 hybrid_clock.cc:648] HybridClock initialized: now 1786021830278614 us; error 31 us; skew 500 ppm
Aug 06 13:10:30 dist-test-slave-q9h9 krb5kdc[9854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021830, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:30.280822 10023 init.cc:377] Logged in from keytab as kudu/127.13.224.65@KRBTEST.COM (short username kudu)
I20260806 13:10:30.281170 10023 webserver.cc:460] Webserver started at http://127.13.224.65:36357/ using document root <none> and password file <none>
I20260806 13:10:30.281340 10023 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:30.281378 10023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:30.281461 10023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:30.282387 10023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "04f511beeac582a48ba8e23878d1f890"
server_key_iv: "4ec4b04a1a862f7593d2aa1d51fa3320"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.282667 10023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "04f511beeac582a48ba8e23878d1f890"
server_key_iv: "4ec4b04a1a862f7593d2aa1d51fa3320"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.283756 10023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:30.284343 10044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:30.284490 10023 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:30.284574 10023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "04f511beeac582a48ba8e23878d1f890"
server_key_iv: "4ec4b04a1a862f7593d2aa1d51fa3320"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.284641 10023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:30.292415 10023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:30.293952 10023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:30.294092 10023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:30.294278 10023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:30.294584 10023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:30.294616 10023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:30.294641 10023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:30.294665 10023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:30.299883 10023 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:40747
I20260806 13:10:30.299944 10209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:40747 every 8 connection(s)
I20260806 13:10:30.300186 10023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:10:30.304877 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 10023
I20260806 13:10:30.304965 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:10:30.305143 14209 external_mini_cluster.cc:1428] Setting key 2edf3b94c0efa88ea182c81252fbd2ba
Aug 06 13:10:30 dist-test-slave-q9h9 krb5kdc[9854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021830, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.65@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:30.308513 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.300797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:42339 (local address 127.13.224.126:43195)
0806 13:10:30.300837 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:30.300839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:30.301030 (+ 191us) server_negotiation.cc:407] Connection header received
0806 13:10:30.301461 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:30.301462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:30.301476 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:30.301529 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:30.302426 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.302917 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.303832 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.304028 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.304082 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.306096 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:30.306097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:30.306098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:30.306104 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:30.307201 (+ 1097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:30.307628 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:30.307629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:30.307629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:30.307692 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:30.307881 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:30.307882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:30.307882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:30.308005 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:30.308067 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:30.308339 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 13:10:30.308427 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:10:30.308571 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.300764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:43195 (local address 127.13.224.65:42339)
0806 13:10:30.300930 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:30.300961 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:30.301372 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:30.301591 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:30.301595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:30.302157 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:30.302337 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.302341 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.302996 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.302997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.303748 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.303751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.304095 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.304096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.304206 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.304574 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:30.304581 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:30.305977 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:30.307272 (+ 1295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:30.307280 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:30.307282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:30.307561 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:30.307751 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:30.307752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:30.307752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:30.307836 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:30.308090 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:30.308096 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:30.308218 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 13:10:30.308306 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 13:10:30.308417 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260806 13:10:30.308990 10210 heartbeater.cc:347] Connected to a master server at 127.13.224.126:43195
I20260806 13:10:30.309077 10210 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:30.309243 10210 heartbeater.cc:511] Master 127.13.224.126:43195 requested a full tablet report, sending...
I20260806 13:10:30.309684 9926 ts_manager.cc:194] Registered new tserver with Master: 85dabe0bd5fd45d6b2cb5b67f1211d6a (127.13.224.65:40747)
I20260806 13:10:30.310302 9926 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.65@KRBTEST.COM'} at 127.13.224.65:42339
WARNING: no policy specified for kudu/127.13.224.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
Entry for principal kudu/127.13.224.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab.
I20260806 13:10:30.328467 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43195
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:30.406127 10218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:30.406292 10218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:30.406313 10218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:30.407367 10218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:30.407400 10218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:30.407423 10218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:10:30.408771 10218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.66.keytab
--principal=kudu/127.13.224.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:30.408937 10218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:30.409096 10218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:30.413524 10230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:30.413573 10226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:30.413569 10218 server_base.cc:1034] running on GCE node
W20260806 13:10:30.413546 10225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:30.413885 10218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:30.414099 10218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:30.415225 10218 hybrid_clock.cc:648] HybridClock initialized: now 1786021830415214 us; error 27 us; skew 500 ppm
Aug 06 13:10:30 dist-test-slave-q9h9 krb5kdc[9854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021830, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:30.417420 10218 init.cc:377] Logged in from keytab as kudu/127.13.224.66@KRBTEST.COM (short username kudu)
I20260806 13:10:30.417773 10218 webserver.cc:460] Webserver started at http://127.13.224.66:45921/ using document root <none> and password file <none>
I20260806 13:10:30.417956 10218 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:30.417994 10218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:30.418120 10218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:30.418990 10218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "867db6d01fbe4171bef45685ac423e80"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "ffd646c5083b23a6699f34b1dfdcce60"
server_key_iv: "f1c57ebe7a9910dcb402b46621027724"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.419258 10218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "867db6d01fbe4171bef45685ac423e80"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "ffd646c5083b23a6699f34b1dfdcce60"
server_key_iv: "f1c57ebe7a9910dcb402b46621027724"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.420454 10218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:30.421037 10238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:30.421167 10218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:30.421247 10218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "867db6d01fbe4171bef45685ac423e80"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "ffd646c5083b23a6699f34b1dfdcce60"
server_key_iv: "f1c57ebe7a9910dcb402b46621027724"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.421303 10218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:30.433174 10218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:30.434818 10218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:30.434937 10218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:30.435108 10218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:30.435357 10218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:30.435389 10218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:30.435415 10218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:30.435439 10218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:30.440773 10218 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:42639
I20260806 13:10:30.440809 10412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:42639 every 8 connection(s)
I20260806 13:10:30.441049 10218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:10:30.442646 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 10218
I20260806 13:10:30.442734 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:10:30.442893 14209 external_mini_cluster.cc:1428] Setting key d5fc6cef2211098c43b51e9bf5f6e44a
Aug 06 13:10:30 dist-test-slave-q9h9 krb5kdc[9854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021830, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.66@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:30.449334 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.441619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:56865 (local address 127.13.224.126:43195)
0806 13:10:30.441669 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:30.441671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:30.441847 (+ 176us) server_negotiation.cc:407] Connection header received
0806 13:10:30.442316 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:30.442318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:30.442330 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:30.442380 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:30.443306 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.443835 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.444748 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.444979 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.445048 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.446924 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:30.446926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:30.446926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:30.446933 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:30.448146 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:30.448490 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:30.448491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:30.448491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:30.448546 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:30.448755 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:30.448756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:30.448757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:30.448863 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:30.448917 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:30.449164 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 13:10:30.449247 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 13:10:30.449450 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.441575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:43195 (local address 127.13.224.66:56865)
0806 13:10:30.441742 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:30.441779 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:30.442225 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:30.442439 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:30.442443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:30.443018 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:30.443225 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.443230 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.443915 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.443916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.444668 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.444670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.445072 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.445073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.445186 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.445585 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:30.445592 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:30.446839 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:30.448218 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:30.448225 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:30.448227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:30.448425 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:30.448599 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:30.448599 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:30.448600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:30.448702 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:30.448917 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:30.448923 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:30.449047 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 13:10:30.449127 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 13:10:30.449264 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260806 13:10:30.449836 10415 heartbeater.cc:347] Connected to a master server at 127.13.224.126:43195
I20260806 13:10:30.449928 10415 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:30.450122 10415 heartbeater.cc:511] Master 127.13.224.126:43195 requested a full tablet report, sending...
I20260806 13:10:30.450456 9926 ts_manager.cc:194] Registered new tserver with Master: 867db6d01fbe4171bef45685ac423e80 (127.13.224.66:42639)
I20260806 13:10:30.450804 9926 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.66@KRBTEST.COM'} at 127.13.224.66:56865
WARNING: no policy specified for kudu/127.13.224.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
Entry for principal kudu/127.13.224.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab.
I20260806 13:10:30.465323 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43195
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:30.542948 10425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:30.543125 10425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:30.543146 10425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:30.544186 10425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:30.544217 10425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:30.544241 10425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:10:30.545601 10425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.67.keytab
--principal=kudu/127.13.224.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:43195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:30.545760 10425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:30.545923 10425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 13:10:30.550513 10425 server_base.cc:1034] running on GCE node
W20260806 13:10:30.550496 10435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:30.550477 10432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:30.550493 10430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:30.550920 10425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:30.551103 10425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:30.552228 10425 hybrid_clock.cc:648] HybridClock initialized: now 1786021830552218 us; error 30 us; skew 500 ppm
Aug 06 13:10:30 dist-test-slave-q9h9 krb5kdc[9854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021830, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:30.554538 10425 init.cc:377] Logged in from keytab as kudu/127.13.224.67@KRBTEST.COM (short username kudu)
I20260806 13:10:30.554950 10425 webserver.cc:460] Webserver started at http://127.13.224.67:39279/ using document root <none> and password file <none>
I20260806 13:10:30.555135 10425 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:30.555179 10425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:30.555301 10425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:30.556202 10425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "c5a6f40fd2a94bf19665c4418ea4c120"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "e01cd10c9ed2bd0464783732d8a56bfe"
server_key_iv: "3b2dea9d78b22dacff463f994eb0bb57"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.556515 10425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "c5a6f40fd2a94bf19665c4418ea4c120"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "e01cd10c9ed2bd0464783732d8a56bfe"
server_key_iv: "3b2dea9d78b22dacff463f994eb0bb57"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.557713 10425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:30.558406 10443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:30.558553 10425 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:30.558632 10425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "c5a6f40fd2a94bf19665c4418ea4c120"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "e01cd10c9ed2bd0464783732d8a56bfe"
server_key_iv: "3b2dea9d78b22dacff463f994eb0bb57"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.558688 10425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:30.581827 10425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:30.583719 10425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:30.583845 10425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:30.584033 10425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:30.584300 10425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:30.584332 10425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:30.584358 10425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:30.584383 10425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:30.589877 10425 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:46727
I20260806 13:10:30.589939 10612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:46727 every 8 connection(s)
I20260806 13:10:30.590214 10425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
Aug 06 13:10:30 dist-test-slave-q9h9 krb5kdc[9854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021830, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.67@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:30.597860 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.590813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:55165 (local address 127.13.224.126:43195)
0806 13:10:30.590863 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:30.590865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:30.591033 (+ 168us) server_negotiation.cc:407] Connection header received
0806 13:10:30.591471 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:30.591472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:30.591481 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:30.591534 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:30.592381 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.592858 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.593717 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.593900 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.593962 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.595906 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:30.595907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:30.595908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:30.595914 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:30.596920 (+ 1006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:30.597231 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:30.597232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:30.597232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:30.597274 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:30.597436 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:30.597437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:30.597437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:30.597523 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:30.597565 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:30.597748 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 13:10:30.597800 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 13:10:30.598006 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.590758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:43195 (local address 127.13.224.67:55165)
0806 13:10:30.590927 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:30.590965 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:30.591395 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:30.591614 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:30.591618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:30.592132 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:30.592311 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.592315 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.592913 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.592914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.593645 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.593647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.593980 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.593981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.594119 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.594515 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:30.594523 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:30.595829 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:30.596979 (+ 1150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:30.596985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:30.596986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:30.597179 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:30.597315 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:30.597316 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:30.597316 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:30.597397 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:30.597562 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:30.597564 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:30.597660 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 13:10:30.597730 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 13:10:30.597839 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260806 13:10:30.598453 10613 heartbeater.cc:347] Connected to a master server at 127.13.224.126:43195
I20260806 13:10:30.598526 10613 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:30.598651 10613 heartbeater.cc:511] Master 127.13.224.126:43195 requested a full tablet report, sending...
I20260806 13:10:30.598948 9926 ts_manager.cc:194] Registered new tserver with Master: c5a6f40fd2a94bf19665c4418ea4c120 (127.13.224.67:46727)
I20260806 13:10:30.599306 9926 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.224.67@KRBTEST.COM'} at 127.13.224.67:55165
I20260806 13:10:30.599496 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 10425
I20260806 13:10:30.599603 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:10:30.599781 14209 external_mini_cluster.cc:1428] Setting key ca36fb26b4f8972e4e521d18f28f41d4
I20260806 13:10:30.600553 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:10:30.606463 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.601726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.13.224.126:43195)
0806 13:10:30.601759 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:30.601760 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:30.601863 (+ 103us) server_negotiation.cc:407] Connection header received
0806 13:10:30.601898 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:30.601899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:30.601906 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:30.601949 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:30.602459 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.602882 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.603519 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.603684 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.603723 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.604689 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:30.604690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:30.604690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:30.604694 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:30.605643 (+ 949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:30.605912 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:30.605913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:30.605913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:30.605951 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:30.606109 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:30.606110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:30.606110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:30.606222 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:30.606287 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:30.606364 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 13:10:30.606412 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 13:10:30.607669 9926 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:10:30.609103 9926 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:10:30.614764 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.610995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:40747 (local address 127.0.0.1:60094)
0806 13:10:30.611235 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:30.611245 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:30.611302 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:30.611584 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:30.611587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:30.611589 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:30.611655 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.611657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.612955 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.612957 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.613763 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.613765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.614558 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.614559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.614609 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.614619 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 13:10:30.614665 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 13:10:30.614697 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":55,"thread_start_us":59,"threads_started":1}
I20260806 13:10:30.614820 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.611253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.13.224.65:40747)
0806 13:10:30.611434 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:30.611438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:30.611448 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:10:30.611484 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:30.611486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:30.611500 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:30.611566 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:30.611736 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.612844 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.613839 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.614489 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.614559 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.614609 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:30.614687 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 13:10:30.614726 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":88,"threads_started":1}
I20260806 13:10:30.615240 10625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.610982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:42639 (local address 127.0.0.1:42468)
0806 13:10:30.611218 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:30.611244 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:30.611292 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:30.611799 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:30.611801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:30.611803 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:30.611853 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.611855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.612710 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.612711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.614116 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.614118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.615033 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.615035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.615094 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.615099 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 13:10:30.615146 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 13:10:30.615181 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":53,"threads_started":1}
I20260806 13:10:30.615500 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.611079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42468 (local address 127.13.224.66:42639)
0806 13:10:30.611587 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:30.611591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:30.611600 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:10:30.611633 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:30.611637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:30.611651 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:30.611716 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:30.611935 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.612619 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.614199 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.614944 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.615006 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.615062 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:30.615388 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 13:10:30.615437 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":76,"threads_started":1}
I20260806 13:10:30.615669 10115 tablet_service.cc:1467] Processing CreateTablet for tablet b11dcad294df408f9ac3e44df9e9f58c (DEFAULT_TABLE table=test-table [id=34cc0624106049a69cda48e4be1a4f02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:30.615994 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.610994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:46727 (local address 127.0.0.1:39056)
0806 13:10:30.611364 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:30.611372 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:30.611406 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:30.611801 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:30.611802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:30.611805 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:30.611854 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.611856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.613035 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.613036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.614106 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.614109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.615780 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.615781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.615843 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.615847 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:10:30.615895 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 13:10:30.615931 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":2,"thread_start_us":66,"threads_started":1}
I20260806 13:10:30.616034 10115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b11dcad294df408f9ac3e44df9e9f58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:30.616200 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.611424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39056 (local address 127.13.224.67:46727)
0806 13:10:30.611554 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:30.611558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:30.611568 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:10:30.611601 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:30.611603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:30.611617 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:30.611692 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:30.611931 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.612943 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.615094 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.615684 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.615753 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.615802 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:30.616090 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 13:10:30.616131 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260806 13:10:30.616739 10517 tablet_service.cc:1467] Processing CreateTablet for tablet b11dcad294df408f9ac3e44df9e9f58c (DEFAULT_TABLE table=test-table [id=34cc0624106049a69cda48e4be1a4f02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:30.616994 10517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b11dcad294df408f9ac3e44df9e9f58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:30.618673 10632 tablet_bootstrap.cc:492] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a: Bootstrap starting.
I20260806 13:10:30.619057 10306 tablet_service.cc:1467] Processing CreateTablet for tablet b11dcad294df408f9ac3e44df9e9f58c (DEFAULT_TABLE table=test-table [id=34cc0624106049a69cda48e4be1a4f02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:30.619400 10306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b11dcad294df408f9ac3e44df9e9f58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:30.619447 10636 tablet_bootstrap.cc:492] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120: Bootstrap starting.
I20260806 13:10:30.619720 10632 tablet_bootstrap.cc:654] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:30.620028 10636 tablet_bootstrap.cc:654] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:30.620299 10632 log.cc:824] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:30.620503 10636 log.cc:824] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:30.621104 10632 tablet_bootstrap.cc:492] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a: No bootstrap required, opened a new log
I20260806 13:10:30.621191 10632 ts_tablet_manager.cc:1397] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 13:10:30.622582 10636 tablet_bootstrap.cc:492] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120: No bootstrap required, opened a new log
I20260806 13:10:30.622645 10636 ts_tablet_manager.cc:1397] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 13:10:30.622854 10641 tablet_bootstrap.cc:492] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80: Bootstrap starting.
I20260806 13:10:30.623070 10632 raft_consensus.cc:348] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867db6d01fbe4171bef45685ac423e80" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42639 } } peers { permanent_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 46727 } } peers { permanent_uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40747 } }
I20260806 13:10:30.623281 10632 raft_consensus.cc:374] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:30.623323 10632 raft_consensus.cc:729] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85dabe0bd5fd45d6b2cb5b67f1211d6a, State: Initialized, Role: FOLLOWER
I20260806 13:10:30.623430 10632 consensus_queue.cc:260] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867db6d01fbe4171bef45685ac423e80" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42639 } } peers { permanent_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 46727 } } peers { permanent_uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40747 } }
I20260806 13:10:30.623477 10641 tablet_bootstrap.cc:654] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:30.623745 10632 ts_tablet_manager.cc:1428] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 13:10:30.623914 10210 heartbeater.cc:503] Master 127.13.224.126:43195 was elected leader, sending a full tablet report...
I20260806 13:10:30.624024 10641 log.cc:824] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:30.624308 10636 raft_consensus.cc:348] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867db6d01fbe4171bef45685ac423e80" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42639 } } peers { permanent_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 46727 } } peers { permanent_uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40747 } }
I20260806 13:10:30.624475 10636 raft_consensus.cc:374] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:30.624557 10636 raft_consensus.cc:729] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a6f40fd2a94bf19665c4418ea4c120, State: Initialized, Role: FOLLOWER
I20260806 13:10:30.624753 10641 tablet_bootstrap.cc:492] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80: No bootstrap required, opened a new log
I20260806 13:10:30.624701 10636 consensus_queue.cc:260] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867db6d01fbe4171bef45685ac423e80" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42639 } } peers { permanent_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 46727 } } peers { permanent_uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40747 } }
I20260806 13:10:30.624814 10641 ts_tablet_manager.cc:1397] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:10:30.624997 10636 ts_tablet_manager.cc:1428] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:10:30.625026 10613 heartbeater.cc:503] Master 127.13.224.126:43195 was elected leader, sending a full tablet report...
I20260806 13:10:30.626420 10641 raft_consensus.cc:348] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867db6d01fbe4171bef45685ac423e80" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42639 } } peers { permanent_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 46727 } } peers { permanent_uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40747 } }
I20260806 13:10:30.626557 10641 raft_consensus.cc:374] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:30.626609 10641 raft_consensus.cc:729] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 867db6d01fbe4171bef45685ac423e80, State: Initialized, Role: FOLLOWER
I20260806 13:10:30.626746 10641 consensus_queue.cc:260] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867db6d01fbe4171bef45685ac423e80" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42639 } } peers { permanent_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 46727 } } peers { permanent_uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40747 } }
I20260806 13:10:30.627010 10641 ts_tablet_manager.cc:1428] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 13:10:30.627060 10415 heartbeater.cc:503] Master 127.13.224.126:43195 was elected leader, sending a full tablet report...
I20260806 13:10:30.671053 10644 raft_consensus.cc:482] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:10:30.671144 10644 raft_consensus.cc:504] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867db6d01fbe4171bef45685ac423e80" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42639 } } peers { permanent_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 46727 } } peers { permanent_uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40747 } }
I20260806 13:10:30.671388 10644 leader_election.cc:288] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 867db6d01fbe4171bef45685ac423e80 (127.13.224.66:42639), 85dabe0bd5fd45d6b2cb5b67f1211d6a (127.13.224.65:40747)
I20260806 13:10:30.673828 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.671513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:42639 (local address 127.13.224.67:53759)
0806 13:10:30.671551 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:30.671561 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:30.671597 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:30.671739 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:30.671741 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:30.671742 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:30.671802 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.671803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.672397 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.672399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.673086 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.673088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.673609 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.673610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.673680 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.673683 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:10:30.673725 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 13:10:30.673763 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 13:10:30.673830 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.671582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:53759 (local address 127.13.224.66:42639)
0806 13:10:30.671633 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:30.671635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:30.671643 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:10:30.671663 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:30.671664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:30.671671 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:30.671719 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:30.671872 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.672327 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.673160 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.673539 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.673598 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.673607 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:30.673736 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 13:10:30.673775 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:10:30.674150 10337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b11dcad294df408f9ac3e44df9e9f58c" candidate_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867db6d01fbe4171bef45685ac423e80" is_pre_election: true
I20260806 13:10:30.674203 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.671569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:46921 (local address 127.13.224.65:40747)
0806 13:10:30.671616 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:30.671618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:30.671846 (+ 228us) server_negotiation.cc:407] Connection header received
0806 13:10:30.671877 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:30.671878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:30.671886 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:30.671938 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:30.672061 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.672494 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.673369 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.673732 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.673791 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.673799 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:30.674102 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 13:10:30.674141 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:10:30.674219 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.671514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:40747 (local address 127.13.224.67:46921)
0806 13:10:30.671779 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:30.671789 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:30.671837 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:30.671940 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:30.671941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:30.671942 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:30.672005 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.672006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.672560 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.672561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.673287 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:30.673289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.673920 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.673921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:30.674007 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.674032 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 13:10:30.674097 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 13:10:30.674142 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":126,"thread_start_us":75,"threads_started":1}
I20260806 13:10:30.674288 10337 raft_consensus.cc:2454] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5a6f40fd2a94bf19665c4418ea4c120 in term 0.
I20260806 13:10:30.674533 10444 leader_election.cc:302] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 867db6d01fbe4171bef45685ac423e80, c5a6f40fd2a94bf19665c4418ea4c120; no voters:
I20260806 13:10:30.674564 10147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b11dcad294df408f9ac3e44df9e9f58c" candidate_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a" is_pre_election: true
I20260806 13:10:30.674656 10644 raft_consensus.cc:2781] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:10:30.674679 10147 raft_consensus.cc:2454] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5a6f40fd2a94bf19665c4418ea4c120 in term 0.
I20260806 13:10:30.674718 10644 raft_consensus.cc:482] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:10:30.674747 10644 raft_consensus.cc:3037] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:30.675362 10644 raft_consensus.cc:504] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867db6d01fbe4171bef45685ac423e80" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42639 } } peers { permanent_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 46727 } } peers { permanent_uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40747 } }
I20260806 13:10:30.675473 10644 leader_election.cc:288] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [CANDIDATE]: Term 1 election: Requested vote from peers 867db6d01fbe4171bef45685ac423e80 (127.13.224.66:42639), 85dabe0bd5fd45d6b2cb5b67f1211d6a (127.13.224.65:40747)
I20260806 13:10:30.675607 10337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b11dcad294df408f9ac3e44df9e9f58c" candidate_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867db6d01fbe4171bef45685ac423e80"
I20260806 13:10:30.675665 10147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b11dcad294df408f9ac3e44df9e9f58c" candidate_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a"
I20260806 13:10:30.675689 10337 raft_consensus.cc:3037] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:30.675735 10147 raft_consensus.cc:3037] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:30.676527 10147 raft_consensus.cc:2454] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5a6f40fd2a94bf19665c4418ea4c120 in term 1.
I20260806 13:10:30.676527 10337 raft_consensus.cc:2454] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5a6f40fd2a94bf19665c4418ea4c120 in term 1.
I20260806 13:10:30.676684 10444 leader_election.cc:302] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 867db6d01fbe4171bef45685ac423e80, c5a6f40fd2a94bf19665c4418ea4c120; no voters:
I20260806 13:10:30.676769 10644 raft_consensus.cc:2781] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:30.676908 10644 raft_consensus.cc:686] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [term 1 LEADER]: Becoming Leader. State: Replica: c5a6f40fd2a94bf19665c4418ea4c120, State: Running, Role: LEADER
I20260806 13:10:30.676985 10644 consensus_queue.cc:237] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867db6d01fbe4171bef45685ac423e80" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42639 } } peers { permanent_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 46727 } } peers { permanent_uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40747 } }
I20260806 13:10:30.677673 9926 catalog_manager.cc:5495] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5a6f40fd2a94bf19665c4418ea4c120 (127.13.224.67). New cstate: current_term: 1 leader_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867db6d01fbe4171bef45685ac423e80" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5a6f40fd2a94bf19665c4418ea4c120" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 46727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40747 } health_report { overall_health: UNKNOWN } } }
I20260806 13:10:30.686051 10631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.684000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39068 (local address 127.13.224.67:46727)
0806 13:10:30.684037 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:30.684039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:30.684046 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:10:30.684091 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:30.684092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:30.684107 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:30.684162 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:30.684278 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.684685 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.685389 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.685563 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.685613 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.685762 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:30.685842 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:30.685904 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:30.685940 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 13:10:30.685957 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:10:30.687505 10337 raft_consensus.cc:1261] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80 [term 1 FOLLOWER]: Refusing update from remote peer c5a6f40fd2a94bf19665c4418ea4c120: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:30.687543 10147 raft_consensus.cc:1261] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a [term 1 FOLLOWER]: Refusing update from remote peer c5a6f40fd2a94bf19665c4418ea4c120: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:30.687850 10648 consensus_queue.cc:1035] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "867db6d01fbe4171bef45685ac423e80" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:30.687958 10644 consensus_queue.cc:1035] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85dabe0bd5fd45d6b2cb5b67f1211d6a" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 40747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:30.690150 10651 mvcc.cc:205] Tried to move back new op lower bound from 7315545418494009344 to 7315545418453544960. Current Snapshot: MvccSnapshot[applied={T|T < 7315545418494009344}]
I20260806 13:10:30.690238 10653 mvcc.cc:205] Tried to move back new op lower bound from 7315545418494009344 to 7315545418453544960. Current Snapshot: MvccSnapshot[applied={T|T < 7315545418494009344}]
I20260806 13:10:30.690521 10652 mvcc.cc:205] Tried to move back new op lower bound from 7315545418494009344 to 7315545418453544960. Current Snapshot: MvccSnapshot[applied={T|T < 7315545418494009344}]
I20260806 13:10:30.692330 9926 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:10:30.692405 9926 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:10:30.693029 9926 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c349be70d0d344f99146ff66ced69e0f: Sending DeleteTablet for 3 replicas of tablet b11dcad294df408f9ac3e44df9e9f58c
I20260806 13:10:30.693334 10517 tablet_service.cc:1514] Processing DeleteTablet for tablet b11dcad294df408f9ac3e44df9e9f58c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:30 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:39056
I20260806 13:10:30.693351 10306 tablet_service.cc:1514] Processing DeleteTablet for tablet b11dcad294df408f9ac3e44df9e9f58c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:30 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:42468
I20260806 13:10:30.693361 10115 tablet_service.cc:1514] Processing DeleteTablet for tablet b11dcad294df408f9ac3e44df9e9f58c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:30 UTC) from {username='kudu', principal='kudu/127.13.224.126@KRBTEST.COM'} at 127.0.0.1:60094
I20260806 13:10:30.693545 10670 tablet_replica.cc:330] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80: stopping tablet replica
I20260806 13:10:30.693555 10672 tablet_replica.cc:330] T b11dcad294df408f9ac3e44df9e9f58c P 85dabe0bd5fd45d6b2cb5b67f1211d6a: stopping tablet replica
I20260806 13:10:30.693629 10670 raft_consensus.cc:2229] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:10:30.693722 10670 raft_consensus.cc:2258] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:30.693857 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 10023
I20260806 13:10:30.696143 10670 ts_tablet_manager.cc:1905] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:10:30.697309 10671 tablet_replica.cc:330] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120: stopping tablet replica
I20260806 13:10:30.697386 10671 raft_consensus.cc:2229] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:10:30.697482 10671 raft_consensus.cc:2258] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:30.699087 10671 ts_tablet_manager.cc:1905] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:10:30.700618 10671 ts_tablet_manager.cc:1918] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:10:30.700730 10671 log.cc:1196] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/b11dcad294df408f9ac3e44df9e9f58c
I20260806 13:10:30.700994 10671 ts_tablet_manager.cc:1939] T b11dcad294df408f9ac3e44df9e9f58c P c5a6f40fd2a94bf19665c4418ea4c120: Deleting consensus metadata
I20260806 13:10:30.705304 9906 catalog_manager.cc:4841] TS c5a6f40fd2a94bf19665c4418ea4c120 (127.13.224.67:46727): tablet b11dcad294df408f9ac3e44df9e9f58c (table test-table [id=34cc0624106049a69cda48e4be1a4f02]) successfully deleted
I20260806 13:10:30.712339 10670 ts_tablet_manager.cc:1918] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:10:30.712407 10670 log.cc:1196] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/b11dcad294df408f9ac3e44df9e9f58c
I20260806 13:10:30.712658 10670 ts_tablet_manager.cc:1939] T b11dcad294df408f9ac3e44df9e9f58c P 867db6d01fbe4171bef45685ac423e80: Deleting consensus metadata
I20260806 13:10:30.713052 9902 catalog_manager.cc:4841] TS 867db6d01fbe4171bef45685ac423e80 (127.13.224.66:42639): tablet b11dcad294df408f9ac3e44df9e9f58c (table test-table [id=34cc0624106049a69cda48e4be1a4f02]) successfully deleted
W20260806 13:10:30.716104 9909 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:40747 (error 108)
I20260806 13:10:30.716312 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 10218
I20260806 13:10:30.718333 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.716441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:40747 (local address 127.0.0.1:60102)
0806 13:10:30.718245 (+ 1804us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:30.718296 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:40747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1781}
W20260806 13:10:30.727630 9909 catalog_manager.cc:4584] TS 85dabe0bd5fd45d6b2cb5b67f1211d6a (127.13.224.65:40747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b11dcad294df408f9ac3e44df9e9f58c: Network error: Client connection negotiation failed: client connection to 127.13.224.65:40747: connect: Connection refused (error 111)
I20260806 13:10:30.731261 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 10425
I20260806 13:10:30.737960 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 9871
2026-08-06T13:10:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1961 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T13:10:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:30Z Disabled control of system clock
I20260806 13:10:30.771891 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:37149
--webserver_interface=127.13.224.126
--webserver_port=0
--builtin_ntp_servers=127.13.224.84:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:37149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:10:30.849514 10680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:30.849706 10680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:30.849725 10680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:30.850801 10680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:30.850834 10680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:30.850845 10680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:30.850857 10680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:30.850869 10680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:30.852102 10680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:37149
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:37149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:30.852283 10680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:30.852497 10680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:30.857012 10687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:30.857018 10691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:30.857038 10680 server_base.cc:1034] running on GCE node
W20260806 13:10:30.857019 10694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:30.857407 10680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:30.857659 10680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:30.858791 10680 hybrid_clock.cc:648] HybridClock initialized: now 1786021830858782 us; error 34 us; skew 500 ppm
I20260806 13:10:30.859863 10680 webserver.cc:460] Webserver started at http://127.13.224.126:40171/ using document root <none> and password file <none>
I20260806 13:10:30.860064 10680 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:30.860106 10680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:30.860227 10680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:30.861183 10680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "5ca57f017d0c4922bbc0e07ae606a0bd"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "745dd456db87c846f6015fc8c2f2e6a3"
server_key_iv: "70f548f4c53790bd3db85bd7b1dc7800"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.861515 10680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "5ca57f017d0c4922bbc0e07ae606a0bd"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "745dd456db87c846f6015fc8c2f2e6a3"
server_key_iv: "70f548f4c53790bd3db85bd7b1dc7800"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.862720 10680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:10:30.863332 10705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:30.863457 10680 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:30.863539 10680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "5ca57f017d0c4922bbc0e07ae606a0bd"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "745dd456db87c846f6015fc8c2f2e6a3"
server_key_iv: "70f548f4c53790bd3db85bd7b1dc7800"
server_key_version: "encryptionkey@0"
I20260806 13:10:30.863595 10680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:30.874504 10680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:30.874753 10680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:30.874884 10680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:30.878686 10680 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:37149
I20260806 13:10:30.878718 10785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:37149 every 8 connection(s)
I20260806 13:10:30.879099 10680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:30.879981 10786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:30.882125 10786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd: Bootstrap starting.
I20260806 13:10:30.882654 10786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:30.882947 10786 log.cc:824] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:30.883543 10786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd: No bootstrap required, opened a new log
I20260806 13:10:30.885042 10786 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca57f017d0c4922bbc0e07ae606a0bd" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37149 } }
I20260806 13:10:30.885167 10786 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:30.885200 10786 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ca57f017d0c4922bbc0e07ae606a0bd, State: Initialized, Role: FOLLOWER
I20260806 13:10:30.885335 10786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca57f017d0c4922bbc0e07ae606a0bd" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37149 } }
I20260806 13:10:30.885414 10786 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:30.885452 10786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:30.885500 10786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:30.885558 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 10680
I20260806 13:10:30.885663 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:30.885856 14209 external_mini_cluster.cc:1428] Setting key 5e77fe7cf1ade26cdc2b75e2e8d8cc89
I20260806 13:10:30.886134 10786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca57f017d0c4922bbc0e07ae606a0bd" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37149 } }
I20260806 13:10:30.886233 10786 leader_election.cc:302] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ca57f017d0c4922bbc0e07ae606a0bd; no voters:
I20260806 13:10:30.886394 10786 leader_election.cc:288] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:30.886436 10791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:30.886580 10791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [term 1 LEADER]: Becoming Leader. State: Replica: 5ca57f017d0c4922bbc0e07ae606a0bd, State: Running, Role: LEADER
I20260806 13:10:30.886629 10786 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:30.886694 10791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca57f017d0c4922bbc0e07ae606a0bd" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37149 } }
I20260806 13:10:30.887012 10791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ca57f017d0c4922bbc0e07ae606a0bd. Latest consensus state: current_term: 1 leader_uuid: "5ca57f017d0c4922bbc0e07ae606a0bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca57f017d0c4922bbc0e07ae606a0bd" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37149 } } }
I20260806 13:10:30.887089 10791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:30.887236 10793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ca57f017d0c4922bbc0e07ae606a0bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca57f017d0c4922bbc0e07ae606a0bd" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 37149 } } }
I20260806 13:10:30.887360 10793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:30.887287 10803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:30.888011 10803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:30.889720 10803 catalog_manager.cc:1347] Generated new cluster ID: b1285fe5791244b8bcdb0fecc6c44c53
I20260806 13:10:30.889776 10803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:30.893667 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:30.886318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58838 (local address 127.13.224.126:37149)
0806 13:10:30.887399 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:30.888603 (+ 1204us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:30.888648 (+ 45us) server_negotiation.cc:407] Connection header received
0806 13:10:30.888756 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:30.888763 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:30.888827 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:30.889033 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:30.889285 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.890505 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.891558 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:30.892139 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:30.892536 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:30.892919 (+ 383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:30.892922 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:30.892922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:30.893109 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:30.893156 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:30.893169 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:30.893377 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:30.893406 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 13:10:30.893476 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1011,"thread_start_us":72,"threads_started":1}
I20260806 13:10:30.896513 10803 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:30.896955 10803 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:30.903622 10803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd: Generated new TSK 0
I20260806 13:10:30.903759 10803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:30.909597 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.65:0
--local_ip_for_outbound_sockets=127.13.224.65
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37149
--builtin_ntp_servers=127.13.224.84:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:10:30.990785 10825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:30.990959 10825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:30.990978 10825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:30.992075 10825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:30.992120 10825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:30.992142 10825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.65
I20260806 13:10:30.993453 10825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.224.65
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.65
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:30.993666 10825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:30.993927 10825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:30.998524 10836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:30.998502 10835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:30.998623 10838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:30.999109 10825 server_base.cc:1034] running on GCE node
I20260806 13:10:30.999284 10825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:30.999477 10825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:31.000620 10825 hybrid_clock.cc:648] HybridClock initialized: now 1786021831000605 us; error 34 us; skew 500 ppm
I20260806 13:10:31.001760 10825 webserver.cc:460] Webserver started at http://127.13.224.65:46735/ using document root <none> and password file <none>
I20260806 13:10:31.001962 10825 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:31.002009 10825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:31.002177 10825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:31.003139 10825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/data/instance:
uuid: "25e4747f87814f43b2f167b18ca488ae"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "6d330e16a18cb2616d6bf687ef8ef6e9"
server_key_iv: "0f583f8f4cf0a8b06269a9f20fa6a8cb"
server_key_version: "encryptionkey@0"
I20260806 13:10:31.003448 10825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance:
uuid: "25e4747f87814f43b2f167b18ca488ae"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "6d330e16a18cb2616d6bf687ef8ef6e9"
server_key_iv: "0f583f8f4cf0a8b06269a9f20fa6a8cb"
server_key_version: "encryptionkey@0"
I20260806 13:10:31.004633 10825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:31.005334 10846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:31.005487 10825 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:31.005569 10825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/wal
uuid: "25e4747f87814f43b2f167b18ca488ae"
format_stamp: "Formatted at 2026-08-06 13:10:30 on dist-test-slave-q9h9"
server_key: "6d330e16a18cb2616d6bf687ef8ef6e9"
server_key_iv: "0f583f8f4cf0a8b06269a9f20fa6a8cb"
server_key_version: "encryptionkey@0"
I20260806 13:10:31.005630 10825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:31.028963 10825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:31.029204 10825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:31.029320 10825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:31.029500 10825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:31.029775 10825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:31.029807 10825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:31.029832 10825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:31.029857 10825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:31.035360 10825 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.65:44565
I20260806 13:10:31.035418 11006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.65:44565 every 8 connection(s)
I20260806 13:10:31.035679 10825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/data/info.pb
I20260806 13:10:31.040298 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.036247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.65:39043 (local address 127.13.224.126:37149)
0806 13:10:31.036299 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:31.036301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:31.036520 (+ 219us) server_negotiation.cc:407] Connection header received
0806 13:10:31.036972 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:31.036974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:31.036988 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:31.037039 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:31.037640 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.038177 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.039134 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.039339 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.039396 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.040047 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:31.040048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:31.040049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:31.040053 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:31.040068 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:31.040073 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:31.040123 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:31.040192 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 13:10:31.040222 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 13:10:31.040468 11010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.036238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37149 (local address 127.13.224.65:39043)
0806 13:10:31.036418 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:31.036449 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:31.036882 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:31.037102 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:31.037105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:31.037115 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:31.037554 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.037559 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.038267 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.038268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.039041 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.039043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.039415 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.039416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.039538 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.039924 (+ 386us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:10:31.039925 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:31.039949 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:10:31.039970 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:31.040123 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:31.040129 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:31.040132 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:10:31.040191 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 13:10:31.040286 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260806 13:10:31.040939 11007 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37149
I20260806 13:10:31.041035 11007 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:31.041195 11007 heartbeater.cc:511] Master 127.13.224.126:37149 requested a full tablet report, sending...
I20260806 13:10:31.041630 10737 ts_manager.cc:194] Registered new tserver with Master: 25e4747f87814f43b2f167b18ca488ae (127.13.224.65:44565)
I20260806 13:10:31.042268 10737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.224.65:39043
I20260806 13:10:31.045073 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 10825
I20260806 13:10:31.045154 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-0/wal/instance
I20260806 13:10:31.045326 14209 external_mini_cluster.cc:1428] Setting key 4719243c8ba6984b4741dcadc5a4dcc3
I20260806 13:10:31.046145 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.66:0
--local_ip_for_outbound_sockets=127.13.224.66
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37149
--builtin_ntp_servers=127.13.224.84:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:10:31.124684 11011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:31.124871 11011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:31.124899 11011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:31.126062 11011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:31.126101 11011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:31.126144 11011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.66
I20260806 13:10:31.127442 11011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.224.66
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.66
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:31.127661 11011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:31.127924 11011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:31.132452 11021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:31.132432 11025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:31.132416 11022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:31.132678 11011 server_base.cc:1034] running on GCE node
I20260806 13:10:31.132881 11011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:31.133109 11011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:31.134240 11011 hybrid_clock.cc:648] HybridClock initialized: now 1786021831134238 us; error 44 us; skew 500 ppm
I20260806 13:10:31.135419 11011 webserver.cc:460] Webserver started at http://127.13.224.66:45139/ using document root <none> and password file <none>
I20260806 13:10:31.135619 11011 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:31.135667 11011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:31.135795 11011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:31.136758 11011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/data/instance:
uuid: "392c8b881fab4370b82c074ea03fbfc7"
format_stamp: "Formatted at 2026-08-06 13:10:31 on dist-test-slave-q9h9"
server_key: "ccb1cfed1052cb4edab0adac19536b0b"
server_key_iv: "c08c77ebe1420168b0bc7328ec0d432a"
server_key_version: "encryptionkey@0"
I20260806 13:10:31.137066 11011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance:
uuid: "392c8b881fab4370b82c074ea03fbfc7"
format_stamp: "Formatted at 2026-08-06 13:10:31 on dist-test-slave-q9h9"
server_key: "ccb1cfed1052cb4edab0adac19536b0b"
server_key_iv: "c08c77ebe1420168b0bc7328ec0d432a"
server_key_version: "encryptionkey@0"
I20260806 13:10:31.138255 11011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:31.138904 11033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:31.139050 11011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:31.139128 11011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/wal
uuid: "392c8b881fab4370b82c074ea03fbfc7"
format_stamp: "Formatted at 2026-08-06 13:10:31 on dist-test-slave-q9h9"
server_key: "ccb1cfed1052cb4edab0adac19536b0b"
server_key_iv: "c08c77ebe1420168b0bc7328ec0d432a"
server_key_version: "encryptionkey@0"
I20260806 13:10:31.139184 11011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:31.165459 11011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:31.165695 11011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:31.165818 11011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:31.166007 11011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:31.166330 11011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:31.166363 11011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:31.166388 11011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:31.166411 11011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:31.171824 11011 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.66:39081
I20260806 13:10:31.171857 11215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.66:39081 every 8 connection(s)
I20260806 13:10:31.172133 11011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/data/info.pb
I20260806 13:10:31.176512 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.172739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.66:37387 (local address 127.13.224.126:37149)
0806 13:10:31.172803 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:31.172805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:31.173002 (+ 197us) server_negotiation.cc:407] Connection header received
0806 13:10:31.173438 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:31.173439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:31.173449 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:31.173497 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:31.174087 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.174565 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.175472 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.175649 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.175691 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.176320 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:31.176322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:31.176322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:31.176326 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:31.176341 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:31.176345 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:31.176381 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:31.176433 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 13:10:31.176456 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 13:10:31.176690 11220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.172723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37149 (local address 127.13.224.66:37387)
0806 13:10:31.172897 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:31.172927 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:31.173362 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:31.173545 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:31.173547 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:31.173555 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:31.173998 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.174002 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.174630 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.174632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.175409 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.175412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.175702 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.175703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.175797 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.176201 (+ 404us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:10:31.176202 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:31.176228 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:10:31.176248 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:31.176383 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:31.176385 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:31.176387 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:10:31.176428 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 13:10:31.176498 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260806 13:10:31.177106 11217 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37149
I20260806 13:10:31.177201 11217 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:31.177368 11217 heartbeater.cc:511] Master 127.13.224.126:37149 requested a full tablet report, sending...
I20260806 13:10:31.177690 10737 ts_manager.cc:194] Registered new tserver with Master: 392c8b881fab4370b82c074ea03fbfc7 (127.13.224.66:39081)
I20260806 13:10:31.178040 10737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.224.66:37387
I20260806 13:10:31.180977 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 11011
I20260806 13:10:31.181066 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/wal/instance
I20260806 13:10:31.181218 14209 external_mini_cluster.cc:1428] Setting key e69be5c73a78e164f09a878633794121
I20260806 13:10:31.181945 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.224.67:0
--local_ip_for_outbound_sockets=127.13.224.67
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37149
--builtin_ntp_servers=127.13.224.84:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 13:10:31.260913 11222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:31.261085 11222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:31.261111 11222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:31.262253 11222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:31.262290 11222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:31.262321 11222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.224.67
I20260806 13:10:31.263625 11222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.224.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.224.67
--webserver_port=0
--tserver_master_addrs=127.13.224.126:37149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.224.67
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:31.263849 11222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:31.264094 11222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:31.268611 11232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:31.268635 11229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:31.268630 11228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:31.268843 11222 server_base.cc:1034] running on GCE node
I20260806 13:10:31.269024 11222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:31.269256 11222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:31.270388 11222 hybrid_clock.cc:648] HybridClock initialized: now 1786021831270372 us; error 30 us; skew 500 ppm
I20260806 13:10:31.271498 11222 webserver.cc:460] Webserver started at http://127.13.224.67:38541/ using document root <none> and password file <none>
I20260806 13:10:31.271718 11222 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:31.271765 11222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:31.271886 11222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:31.272821 11222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/data/instance:
uuid: "6966e1b707164d06af2cf516da0c7b41"
format_stamp: "Formatted at 2026-08-06 13:10:31 on dist-test-slave-q9h9"
server_key: "e94f8d99d2e67d804617f2c3d0bddf0d"
server_key_iv: "98e97bf66acbdc87d8309df8ce4822a6"
server_key_version: "encryptionkey@0"
I20260806 13:10:31.273118 11222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance:
uuid: "6966e1b707164d06af2cf516da0c7b41"
format_stamp: "Formatted at 2026-08-06 13:10:31 on dist-test-slave-q9h9"
server_key: "e94f8d99d2e67d804617f2c3d0bddf0d"
server_key_iv: "98e97bf66acbdc87d8309df8ce4822a6"
server_key_version: "encryptionkey@0"
I20260806 13:10:31.274305 11222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:31.274962 11238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:31.275101 11222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:10:31.275192 11222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/wal
uuid: "6966e1b707164d06af2cf516da0c7b41"
format_stamp: "Formatted at 2026-08-06 13:10:31 on dist-test-slave-q9h9"
server_key: "e94f8d99d2e67d804617f2c3d0bddf0d"
server_key_iv: "98e97bf66acbdc87d8309df8ce4822a6"
server_key_version: "encryptionkey@0"
I20260806 13:10:31.275250 11222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:31.291649 11222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:31.291872 11222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:31.291985 11222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:31.292167 11222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 13:10:31.292433 11222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 13:10:31.292465 11222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:31.292490 11222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 13:10:31.292515 11222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:31.297902 11222 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.67:36295
I20260806 13:10:31.297940 11413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.67:36295 every 8 connection(s)
I20260806 13:10:31.298238 11222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/data/info.pb
I20260806 13:10:31.302574 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.298854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:34777 (local address 127.13.224.126:37149)
0806 13:10:31.298902 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:31.298904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:31.299104 (+ 200us) server_negotiation.cc:407] Connection header received
0806 13:10:31.299513 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:31.299514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:31.299524 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:31.299570 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:31.300140 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.300609 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.301475 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.301650 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.301692 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.302387 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:31.302388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:31.302388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:31.302392 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:31.302405 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:31.302409 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:31.302447 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:31.302495 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 13:10:31.302519 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:10:31.302742 11418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.298835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:37149 (local address 127.13.224.67:34777)
0806 13:10:31.299002 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:31.299031 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:31.299441 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:31.299615 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:31.299618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:31.299625 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:31.300068 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.300072 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.300669 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.300670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.301418 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.301420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.301739 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.301740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.301860 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.302271 (+ 411us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 13:10:31.302271 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:31.302295 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 13:10:31.302315 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:31.302447 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:31.302449 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:31.302451 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 13:10:31.302492 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 13:10:31.302562 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260806 13:10:31.303179 11416 heartbeater.cc:347] Connected to a master server at 127.13.224.126:37149
I20260806 13:10:31.303260 11416 heartbeater.cc:464] Registering TS with master...
I20260806 13:10:31.303418 11416 heartbeater.cc:511] Master 127.13.224.126:37149 requested a full tablet report, sending...
I20260806 13:10:31.303735 10737 ts_manager.cc:194] Registered new tserver with Master: 6966e1b707164d06af2cf516da0c7b41 (127.13.224.67:36295)
I20260806 13:10:31.304073 10737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.224.67:34777
I20260806 13:10:31.306707 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 11222
I20260806 13:10:31.306792 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/wal/instance
I20260806 13:10:31.306931 14209 external_mini_cluster.cc:1428] Setting key c365a7b3f8cc57aa6c3dd8e9fa97f527
I20260806 13:10:31.307680 14209 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 13:10:31.311622 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.308904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58848 (local address 127.13.224.126:37149)
0806 13:10:31.308948 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:31.308949 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:31.309051 (+ 102us) server_negotiation.cc:407] Connection header received
0806 13:10:31.309111 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:31.309112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:31.309137 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:31.309182 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:31.309360 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.309791 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.310638 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.310835 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.310886 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.311426 (+ 540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:31.311427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:31.311427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:31.311431 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:31.311443 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:31.311447 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:31.311485 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:31.311535 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 13:10:31.311557 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 13:10:31.312909 10737 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 13:10:31.314360 10737 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 13:10:31.319850 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.316136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.67:36295 (local address 127.0.0.1:37858)
0806 13:10:31.316653 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:31.316663 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:31.316715 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:31.316998 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:31.316999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:31.317001 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:31.317044 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.317046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.318187 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.318188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.318921 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.318924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.319568 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.319569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.319654 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.319669 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 13:10:31.319731 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 13:10:31.319781 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":78,"threads_started":1}
I20260806 13:10:31.319862 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.316306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37858 (local address 127.13.224.67:36295)
0806 13:10:31.316441 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:31.316445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:31.316716 (+ 271us) server_negotiation.cc:407] Connection header received
0806 13:10:31.316790 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:31.316792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:31.316805 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:31.316879 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:31.317117 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.318119 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.319003 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.319489 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.319550 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.319627 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:31.319759 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 13:10:31.319790 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260806 13:10:31.320218 11427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.316118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:44565 (local address 127.0.0.1:50518)
0806 13:10:31.316270 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:31.316305 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:31.316365 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:31.316550 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:31.316553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:31.316555 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:31.316605 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.316606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.317679 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.317679 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.318900 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.318902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.319881 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.319882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.320060 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.320064 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 13:10:31.320118 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 13:10:31.320153 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":42,"threads_started":1}
I20260806 13:10:31.320272 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.316200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50518 (local address 127.13.224.65:44565)
0806 13:10:31.316384 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:31.316388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:31.316400 (+ 12us) server_negotiation.cc:407] Connection header received
0806 13:10:31.316443 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:31.316445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:31.316459 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:31.316546 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:31.316775 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.317602 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.318997 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.319704 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.319765 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.319844 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:31.320148 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 13:10:31.320193 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":76,"threads_started":1}
I20260806 13:10:31.320436 11426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.316133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:39081 (local address 127.0.0.1:58132)
0806 13:10:31.316464 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:31.316472 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:31.316506 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:31.316662 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:31.316663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:31.316665 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:31.316697 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.316697 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.317891 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.317892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.318945 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.318946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.319882 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.319883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.320168 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.320171 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:10:31.320225 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 13:10:31.320259 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260806 13:10:31.320492 11305 tablet_service.cc:1467] Processing CreateTablet for tablet e74ac50968264f2ab07abee0a670f4c4 (DEFAULT_TABLE table=test-table [id=c80e524979234fb18431bcfa0803c7ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:31.320709 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.316354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58132 (local address 127.13.224.66:39081)
0806 13:10:31.316524 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:31.316527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:31.316537 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:10:31.316569 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:31.316571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:31.316584 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:31.316657 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:31.316757 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.317812 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.319024 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.319697 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.319765 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.319843 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:31.320592 (+ 749us) server_negotiation.cc:299] Negotiation successful
0806 13:10:31.320631 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":77,"threads_started":1}
I20260806 13:10:31.320819 11305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e74ac50968264f2ab07abee0a670f4c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:31.321252 11108 tablet_service.cc:1467] Processing CreateTablet for tablet e74ac50968264f2ab07abee0a670f4c4 (DEFAULT_TABLE table=test-table [id=c80e524979234fb18431bcfa0803c7ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:31.321528 11108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e74ac50968264f2ab07abee0a670f4c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:31.322428 10909 tablet_service.cc:1467] Processing CreateTablet for tablet e74ac50968264f2ab07abee0a670f4c4 (DEFAULT_TABLE table=test-table [id=c80e524979234fb18431bcfa0803c7ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 13:10:31.322757 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e74ac50968264f2ab07abee0a670f4c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:31.323765 11432 tablet_bootstrap.cc:492] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41: Bootstrap starting.
I20260806 13:10:31.324379 11432 tablet_bootstrap.cc:654] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:31.324646 11432 log.cc:824] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:31.324961 11433 tablet_bootstrap.cc:492] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7: Bootstrap starting.
I20260806 13:10:31.325359 11432 tablet_bootstrap.cc:492] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41: No bootstrap required, opened a new log
I20260806 13:10:31.325423 11432 ts_tablet_manager.cc:1397] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:10:31.325573 11433 tablet_bootstrap.cc:654] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:31.325584 11435 tablet_bootstrap.cc:492] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae: Bootstrap starting.
I20260806 13:10:31.325886 11433 log.cc:824] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:31.326293 11435 tablet_bootstrap.cc:654] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:31.326570 11435 log.cc:824] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:31.326629 11433 tablet_bootstrap.cc:492] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7: No bootstrap required, opened a new log
I20260806 13:10:31.326706 11433 ts_tablet_manager.cc:1397] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 13:10:31.327178 11435 tablet_bootstrap.cc:492] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae: No bootstrap required, opened a new log
I20260806 13:10:31.327232 11435 ts_tablet_manager.cc:1397] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 13:10:31.327826 11432 raft_consensus.cc:348] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e4747f87814f43b2f167b18ca488ae" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44565 } } peers { permanent_uuid: "392c8b881fab4370b82c074ea03fbfc7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39081 } } peers { permanent_uuid: "6966e1b707164d06af2cf516da0c7b41" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 36295 } }
I20260806 13:10:31.327984 11432 raft_consensus.cc:374] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:31.328020 11432 raft_consensus.cc:729] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6966e1b707164d06af2cf516da0c7b41, State: Initialized, Role: FOLLOWER
I20260806 13:10:31.328146 11432 consensus_queue.cc:260] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e4747f87814f43b2f167b18ca488ae" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44565 } } peers { permanent_uuid: "392c8b881fab4370b82c074ea03fbfc7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39081 } } peers { permanent_uuid: "6966e1b707164d06af2cf516da0c7b41" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 36295 } }
I20260806 13:10:31.328416 11432 ts_tablet_manager.cc:1428] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 13:10:31.328524 11416 heartbeater.cc:503] Master 127.13.224.126:37149 was elected leader, sending a full tablet report...
I20260806 13:10:31.329061 11433 raft_consensus.cc:348] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e4747f87814f43b2f167b18ca488ae" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44565 } } peers { permanent_uuid: "392c8b881fab4370b82c074ea03fbfc7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39081 } } peers { permanent_uuid: "6966e1b707164d06af2cf516da0c7b41" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 36295 } }
I20260806 13:10:31.329211 11433 raft_consensus.cc:374] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:31.329249 11433 raft_consensus.cc:729] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 392c8b881fab4370b82c074ea03fbfc7, State: Initialized, Role: FOLLOWER
I20260806 13:10:31.329258 11435 raft_consensus.cc:348] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e4747f87814f43b2f167b18ca488ae" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44565 } } peers { permanent_uuid: "392c8b881fab4370b82c074ea03fbfc7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39081 } } peers { permanent_uuid: "6966e1b707164d06af2cf516da0c7b41" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 36295 } }
I20260806 13:10:31.329368 11433 consensus_queue.cc:260] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e4747f87814f43b2f167b18ca488ae" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44565 } } peers { permanent_uuid: "392c8b881fab4370b82c074ea03fbfc7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39081 } } peers { permanent_uuid: "6966e1b707164d06af2cf516da0c7b41" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 36295 } }
I20260806 13:10:31.329406 11435 raft_consensus.cc:374] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:31.329443 11435 raft_consensus.cc:729] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e4747f87814f43b2f167b18ca488ae, State: Initialized, Role: FOLLOWER
I20260806 13:10:31.329610 11433 ts_tablet_manager.cc:1428] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 13:10:31.329663 11217 heartbeater.cc:503] Master 127.13.224.126:37149 was elected leader, sending a full tablet report...
I20260806 13:10:31.329546 11435 consensus_queue.cc:260] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e4747f87814f43b2f167b18ca488ae" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44565 } } peers { permanent_uuid: "392c8b881fab4370b82c074ea03fbfc7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39081 } } peers { permanent_uuid: "6966e1b707164d06af2cf516da0c7b41" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 36295 } }
I20260806 13:10:31.329981 11007 heartbeater.cc:503] Master 127.13.224.126:37149 was elected leader, sending a full tablet report...
I20260806 13:10:31.329928 11435 ts_tablet_manager.cc:1428] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260806 13:10:31.422612 11219 tablet.cc:2369] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 13:10:31.446281 11439 raft_consensus.cc:482] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 13:10:31.446374 11439 raft_consensus.cc:504] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e4747f87814f43b2f167b18ca488ae" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44565 } } peers { permanent_uuid: "392c8b881fab4370b82c074ea03fbfc7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39081 } } peers { permanent_uuid: "6966e1b707164d06af2cf516da0c7b41" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 36295 } }
I20260806 13:10:31.446656 11439 leader_election.cc:288] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25e4747f87814f43b2f167b18ca488ae (127.13.224.65:44565), 392c8b881fab4370b82c074ea03fbfc7 (127.13.224.66:39081)
I20260806 13:10:31.449178 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.446859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:39079 (local address 127.13.224.65:44565)
0806 13:10:31.446898 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:31.446900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:31.446910 (+ 10us) server_negotiation.cc:407] Connection header received
0806 13:10:31.446985 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:31.446986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:31.446996 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:31.447035 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:31.447168 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.447650 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.448543 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.448886 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.448947 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.448955 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:31.449081 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 13:10:31.449113 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:10:31.449178 11418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.446812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:44565 (local address 127.13.224.67:39079)
0806 13:10:31.446875 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:31.446886 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:31.446925 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:31.447046 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:31.447047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:31.447049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:31.447111 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.447113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.447729 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.447731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.448459 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.448461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.448958 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.448960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.449021 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.449024 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:10:31.449069 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 13:10:31.449111 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 13:10:31.449321 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.446885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.224.67:59141 (local address 127.13.224.66:39081)
0806 13:10:31.446927 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:31.446929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:31.447070 (+ 141us) server_negotiation.cc:407] Connection header received
0806 13:10:31.447119 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:31.447120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:31.447129 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:31.447173 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 13:10:31.447319 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.447773 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.448655 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.449003 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.449061 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.449069 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:31.449215 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 13:10:31.449251 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:10:31.449494 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.446846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.66:39081 (local address 127.13.224.67:59141)
0806 13:10:31.447002 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:31.447011 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:31.447067 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:31.447182 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:31.447183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:31.447184 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 13:10:31.447263 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.447264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.447846 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.447847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.448573 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:31.448575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.449075 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.449076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:31.449155 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.449158 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 13:10:31.449201 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 13:10:31.449415 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260806 13:10:31.449489 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74ac50968264f2ab07abee0a670f4c4" candidate_uuid: "6966e1b707164d06af2cf516da0c7b41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e4747f87814f43b2f167b18ca488ae" is_pre_election: true
I20260806 13:10:31.449622 10939 raft_consensus.cc:2454] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6966e1b707164d06af2cf516da0c7b41 in term 0.
I20260806 13:10:31.449822 11139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74ac50968264f2ab07abee0a670f4c4" candidate_uuid: "6966e1b707164d06af2cf516da0c7b41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "392c8b881fab4370b82c074ea03fbfc7" is_pre_election: true
I20260806 13:10:31.449846 11239 leader_election.cc:302] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e4747f87814f43b2f167b18ca488ae, 6966e1b707164d06af2cf516da0c7b41; no voters:
I20260806 13:10:31.449929 11139 raft_consensus.cc:2454] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6966e1b707164d06af2cf516da0c7b41 in term 0.
I20260806 13:10:31.449973 11439 raft_consensus.cc:2781] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 13:10:31.450062 11439 raft_consensus.cc:482] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 13:10:31.450086 11439 raft_consensus.cc:3037] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:31.450764 11439 raft_consensus.cc:504] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e4747f87814f43b2f167b18ca488ae" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44565 } } peers { permanent_uuid: "392c8b881fab4370b82c074ea03fbfc7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39081 } } peers { permanent_uuid: "6966e1b707164d06af2cf516da0c7b41" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 36295 } }
I20260806 13:10:31.450870 11439 leader_election.cc:288] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [CANDIDATE]: Term 1 election: Requested vote from peers 25e4747f87814f43b2f167b18ca488ae (127.13.224.65:44565), 392c8b881fab4370b82c074ea03fbfc7 (127.13.224.66:39081)
I20260806 13:10:31.451018 10939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74ac50968264f2ab07abee0a670f4c4" candidate_uuid: "6966e1b707164d06af2cf516da0c7b41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e4747f87814f43b2f167b18ca488ae"
I20260806 13:10:31.451021 11139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e74ac50968264f2ab07abee0a670f4c4" candidate_uuid: "6966e1b707164d06af2cf516da0c7b41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "392c8b881fab4370b82c074ea03fbfc7"
I20260806 13:10:31.451092 11139 raft_consensus.cc:3037] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:31.451092 10939 raft_consensus.cc:3037] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:31.451927 11139 raft_consensus.cc:2454] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6966e1b707164d06af2cf516da0c7b41 in term 1.
I20260806 13:10:31.451927 10939 raft_consensus.cc:2454] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6966e1b707164d06af2cf516da0c7b41 in term 1.
I20260806 13:10:31.452095 11239 leader_election.cc:302] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e4747f87814f43b2f167b18ca488ae, 6966e1b707164d06af2cf516da0c7b41; no voters:
I20260806 13:10:31.452203 11439 raft_consensus.cc:2781] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:31.452337 11439 raft_consensus.cc:686] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [term 1 LEADER]: Becoming Leader. State: Replica: 6966e1b707164d06af2cf516da0c7b41, State: Running, Role: LEADER
I20260806 13:10:31.452486 11439 consensus_queue.cc:237] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e4747f87814f43b2f167b18ca488ae" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44565 } } peers { permanent_uuid: "392c8b881fab4370b82c074ea03fbfc7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39081 } } peers { permanent_uuid: "6966e1b707164d06af2cf516da0c7b41" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 36295 } }
I20260806 13:10:31.453229 10737 catalog_manager.cc:5495] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6966e1b707164d06af2cf516da0c7b41 (127.13.224.67). New cstate: current_term: 1 leader_uuid: "6966e1b707164d06af2cf516da0c7b41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25e4747f87814f43b2f167b18ca488ae" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "392c8b881fab4370b82c074ea03fbfc7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6966e1b707164d06af2cf516da0c7b41" member_type: VOTER last_known_addr { host: "127.13.224.67" port: 36295 } health_report { overall_health: HEALTHY } } }
I20260806 13:10:31.463129 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.461070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37874 (local address 127.13.224.67:36295)
0806 13:10:31.461124 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:31.461126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:31.461134 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:10:31.461164 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:31.461165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:31.461179 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:31.461249 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:31.461352 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.461766 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.462482 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:31.462655 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:31.462705 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:31.462852 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:31.462934 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:31.463000 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:31.463037 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 13:10:31.463052 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 13:10:31.464560 11139 raft_consensus.cc:1261] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7 [term 1 FOLLOWER]: Refusing update from remote peer 6966e1b707164d06af2cf516da0c7b41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:31.464560 10939 raft_consensus.cc:1261] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae [term 1 FOLLOWER]: Refusing update from remote peer 6966e1b707164d06af2cf516da0c7b41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:31.464915 11445 consensus_queue.cc:1035] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "392c8b881fab4370b82c074ea03fbfc7" member_type: VOTER last_known_addr { host: "127.13.224.66" port: 39081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:31.465037 11439 consensus_queue.cc:1035] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e4747f87814f43b2f167b18ca488ae" member_type: VOTER last_known_addr { host: "127.13.224.65" port: 44565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:31.467206 11450 mvcc.cc:205] Tried to move back new op lower bound from 7315545421676863488 to 7315545421630050304. Current Snapshot: MvccSnapshot[applied={T|T < 7315545421676863488}]
I20260806 13:10:31.467523 11451 mvcc.cc:205] Tried to move back new op lower bound from 7315545421676863488 to 7315545421630050304. Current Snapshot: MvccSnapshot[applied={T|T < 7315545421676863488}]
I20260806 13:10:31.467844 11452 mvcc.cc:205] Tried to move back new op lower bound from 7315545421676863488 to 7315545421630050304. Current Snapshot: MvccSnapshot[applied={T|T < 7315545421676863488}]
I20260806 13:10:31.470258 10737 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:10:31.470341 10737 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 13:10:31.470999 10737 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ca57f017d0c4922bbc0e07ae606a0bd: Sending DeleteTablet for 3 replicas of tablet e74ac50968264f2ab07abee0a670f4c4
I20260806 13:10:31.471320 11108 tablet_service.cc:1514] Processing DeleteTablet for tablet e74ac50968264f2ab07abee0a670f4c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:31 UTC) from {username='slave'} at 127.0.0.1:58132
I20260806 13:10:31.471318 10909 tablet_service.cc:1514] Processing DeleteTablet for tablet e74ac50968264f2ab07abee0a670f4c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:31 UTC) from {username='slave'} at 127.0.0.1:50518
I20260806 13:10:31.471541 11461 tablet_replica.cc:330] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae: stopping tablet replica
I20260806 13:10:31.471527 11305 tablet_service.cc:1514] Processing DeleteTablet for tablet e74ac50968264f2ab07abee0a670f4c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 13:10:31 UTC) from {username='slave'} at 127.0.0.1:37858
I20260806 13:10:31.471668 11461 raft_consensus.cc:2229] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:10:31.471735 11464 tablet_replica.cc:330] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41: stopping tablet replica
I20260806 13:10:31.471809 11461 raft_consensus.cc:2258] T e74ac50968264f2ab07abee0a670f4c4 P 25e4747f87814f43b2f167b18ca488ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:31.471827 11464 raft_consensus.cc:2229] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [term 1 LEADER]: Raft consensus shutting down.
I20260806 13:10:31.471939 11464 raft_consensus.cc:2258] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:31.471944 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 10825
I20260806 13:10:31.472673 11464 ts_tablet_manager.cc:1905] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:10:31.474483 11464 ts_tablet_manager.cc:1918] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:10:31.474553 11464 log.cc:1196] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-2/wal/wals/e74ac50968264f2ab07abee0a670f4c4
I20260806 13:10:31.474839 11464 ts_tablet_manager.cc:1939] T e74ac50968264f2ab07abee0a670f4c4 P 6966e1b707164d06af2cf516da0c7b41: Deleting consensus metadata
I20260806 13:10:31.476176 10719 catalog_manager.cc:4841] TS 6966e1b707164d06af2cf516da0c7b41 (127.13.224.67:36295): tablet e74ac50968264f2ab07abee0a670f4c4 (table test-table [id=c80e524979234fb18431bcfa0803c7ee]) successfully deleted
I20260806 13:10:31.482122 11463 tablet_replica.cc:330] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7: stopping tablet replica
I20260806 13:10:31.482237 11463 raft_consensus.cc:2229] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 13:10:31.482364 11463 raft_consensus.cc:2258] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 13:10:31.484045 11463 ts_tablet_manager.cc:1905] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 13:10:31.485940 11463 ts_tablet_manager.cc:1918] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 13:10:31.486054 11463 log.cc:1196] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7: Deleting WAL directory at /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786021591455662-14209-0/minicluster-data/ts-1/wal/wals/e74ac50968264f2ab07abee0a670f4c4
I20260806 13:10:31.486378 11463 ts_tablet_manager.cc:1939] T e74ac50968264f2ab07abee0a670f4c4 P 392c8b881fab4370b82c074ea03fbfc7: Deleting consensus metadata
I20260806 13:10:31.487159 10717 catalog_manager.cc:4841] TS 392c8b881fab4370b82c074ea03fbfc7 (127.13.224.66:39081): tablet e74ac50968264f2ab07abee0a670f4c4 (table test-table [id=c80e524979234fb18431bcfa0803c7ee]) successfully deleted
W20260806 13:10:31.490643 10707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.224.65:44565 (error 108)
I20260806 13:10:31.491110 11426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:31.490986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.65:44565 (local address 127.0.0.1:50532)
0806 13:10:31.491036 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:31.491077 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.65:44565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 13:10:31.491149 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 11011
W20260806 13:10:31.495112 10707 catalog_manager.cc:4584] TS 25e4747f87814f43b2f167b18ca488ae (127.13.224.65:44565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e74ac50968264f2ab07abee0a670f4c4: Network error: Client connection negotiation failed: client connection to 127.13.224.65:44565: connect: Connection refused (error 111)
I20260806 13:10:31.498425 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 11222
I20260806 13:10:31.507539 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 10680
2026-08-06T13:10:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (767 ms)
[----------] 6 tests from EncryptionPolicyTest (8780 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:31 dist-test-slave-q9h9 krb5kdc[11469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:31 dist-test-slave-q9h9 krb5kdc[11469](info): set up 2 sockets
Aug 06 13:10:31 dist-test-slave-q9h9 krb5kdc[11469](info): commencing operation
krb5kdc: starting...
W20260806 13:10:32.607908 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:32 dist-test-slave-q9h9 krb5kdc[11469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:10:32.714265 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42741
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:32.791971 11488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:32.792147 11488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:32.792168 11488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:32.793243 11488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:32.793277 11488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:32.793289 11488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:32.793300 11488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:32.793313 11488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:32.794711 11488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42741
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:32.794906 11488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:32.795076 11488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:32.799494 11502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:32.799623 11488 server_base.cc:1034] running on GCE node
W20260806 13:10:32.799505 11505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:32.799494 11509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:32.799947 11488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:32.800211 11488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:32.801350 11488 hybrid_clock.cc:648] HybridClock initialized: now 1786021832801336 us; error 26 us; skew 500 ppm
Aug 06 13:10:32 dist-test-slave-q9h9 krb5kdc[11469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021832, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:32.803588 11488 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:10:32.803988 11488 webserver.cc:460] Webserver started at http://127.0.0.1:41947/ using document root <none> and password file <none>
I20260806 13:10:32.804201 11488 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:32.804247 11488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:32.804371 11488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:32.805305 11488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "d6d51b7ffda249bf9c5150ab2aa91662"
format_stamp: "Formatted at 2026-08-06 13:10:32 on dist-test-slave-q9h9"
server_key: "66fe39d5b9b6797d472cb44eef3dcce9"
server_key_iv: "82411f7b74faea21481eba25ad9b3a07"
server_key_version: "encryptionkey@0"
I20260806 13:10:32.805615 11488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "d6d51b7ffda249bf9c5150ab2aa91662"
format_stamp: "Formatted at 2026-08-06 13:10:32 on dist-test-slave-q9h9"
server_key: "66fe39d5b9b6797d472cb44eef3dcce9"
server_key_iv: "82411f7b74faea21481eba25ad9b3a07"
server_key_version: "encryptionkey@0"
I20260806 13:10:32.806830 11488 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:10:32.807469 11519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:32.807613 11488 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:10:32.807696 11488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "d6d51b7ffda249bf9c5150ab2aa91662"
format_stamp: "Formatted at 2026-08-06 13:10:32 on dist-test-slave-q9h9"
server_key: "66fe39d5b9b6797d472cb44eef3dcce9"
server_key_iv: "82411f7b74faea21481eba25ad9b3a07"
server_key_version: "encryptionkey@0"
I20260806 13:10:32.807754 11488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:32.814877 11488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:32.816344 11488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:32.816462 11488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:32.820087 11488 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42741
I20260806 13:10:32.820120 11603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42741 every 8 connection(s)
I20260806 13:10:32.820422 11488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:32.821182 11604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:32.823191 11604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662: Bootstrap starting.
I20260806 13:10:32.823684 11604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:32.823915 11604 log.cc:824] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:32.824512 11604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662: No bootstrap required, opened a new log
I20260806 13:10:32.825937 11604 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6d51b7ffda249bf9c5150ab2aa91662" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42741 } }
I20260806 13:10:32.826097 11604 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:32.826124 11604 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6d51b7ffda249bf9c5150ab2aa91662, State: Initialized, Role: FOLLOWER
I20260806 13:10:32.826229 11604 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6d51b7ffda249bf9c5150ab2aa91662" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42741 } }
I20260806 13:10:32.826309 11604 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:32.826336 11604 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:32.826367 11604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:32.826788 11604 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6d51b7ffda249bf9c5150ab2aa91662" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42741 } }
I20260806 13:10:32.826869 11604 leader_election.cc:302] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6d51b7ffda249bf9c5150ab2aa91662; no voters:
I20260806 13:10:32.826980 11604 leader_election.cc:288] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:32.827037 11608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:32.827183 11604 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:32.827188 11608 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [term 1 LEADER]: Becoming Leader. State: Replica: d6d51b7ffda249bf9c5150ab2aa91662, State: Running, Role: LEADER
I20260806 13:10:32.827293 11608 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6d51b7ffda249bf9c5150ab2aa91662" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42741 } }
I20260806 13:10:32.827680 11610 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6d51b7ffda249bf9c5150ab2aa91662. Latest consensus state: current_term: 1 leader_uuid: "d6d51b7ffda249bf9c5150ab2aa91662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6d51b7ffda249bf9c5150ab2aa91662" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42741 } } }
I20260806 13:10:32.827749 11609 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6d51b7ffda249bf9c5150ab2aa91662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6d51b7ffda249bf9c5150ab2aa91662" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42741 } } }
I20260806 13:10:32.827872 11609 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:32.827936 11610 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:32.827927 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 11488
I20260806 13:10:32.828027 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:32.828210 14209 external_mini_cluster.cc:1428] Setting key 4cd413ff939c53576d069e64c517e6c3
I20260806 13:10:32.828575 11622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:32.829095 11622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:32.830559 11622 catalog_manager.cc:1347] Generated new cluster ID: 5c42af710db64a559af167924dc9b952
I20260806 13:10:32.830613 11622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:10:32 dist-test-slave-q9h9 krb5kdc[11469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 13:10:32.837817 11622 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:32.838413 11622 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:32.839450 11629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:32.828817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36331 (local address 127.0.0.1:42741)
0806 13:10:32.828991 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:32.828999 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:32.829026 (+ 27us) server_negotiation.cc:407] Connection header received
0806 13:10:32.829135 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:32.829141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:32.829198 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:32.830111 (+ 913us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:32.830195 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:32.831533 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:32.832585 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:32.833144 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:32.833232 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:32.835840 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:32.835845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:32.835846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:32.835859 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:32.837738 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:32.838268 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:32.838271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:32.838272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:32.838376 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:32.838625 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:32.838626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:32.838626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:32.838896 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:32.839066 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:32.839153 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 13:10:32.839252 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260806 13:10:32.840246 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:32.841246 11622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6d51b7ffda249bf9c5150ab2aa91662: Generated new TSK 0
I20260806 13:10:32.841391 11622 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:32.847537 11629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:32.841710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54425 (local address 127.0.0.1:42741)
0806 13:10:32.841766 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:32.841768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:32.841801 (+ 33us) server_negotiation.cc:407] Connection header received
0806 13:10:32.841858 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:32.841858 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:32.841865 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:32.841906 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:32.842499 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:32.842970 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:32.843941 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:32.844193 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:32.844240 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:32.845392 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:32.845393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:32.845393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:32.845399 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:32.846584 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:32.846951 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:32.846952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:32.846952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:32.846992 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:32.847169 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:32.847170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:32.847170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:32.847247 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:32.847316 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:32.847447 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 13:10:32.847487 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:10:32.849582 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 11488
W20260806 13:10:32.860816 11467 connection.cc:535] client connection to 127.0.0.1:42741 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42741): Connection reset by peer (error 104)
2026-08-06T13:10:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:32 dist-test-slave-q9h9 krb5kdc[11652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:32 dist-test-slave-q9h9 krb5kdc[11652](info): set up 2 sockets
Aug 06 13:10:32 dist-test-slave-q9h9 krb5kdc[11652](info): commencing operation
krb5kdc: starting...
W20260806 13:10:33.980810 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:34 dist-test-slave-q9h9 krb5kdc[11652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:10:34.090620 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42963
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:34.169638 11669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:34.169837 11669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:34.169869 11669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:34.170986 11669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:34.171025 11669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:34.171043 11669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:34.171059 11669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:34.171074 11669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:34.172504 11669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42963
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:34.172736 11669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:34.172945 11669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:34.177446 11682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:34.177434 11679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:34.177559 11669 server_base.cc:1034] running on GCE node
W20260806 13:10:34.177423 11675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:34.177850 11669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:34.178133 11669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:34.179279 11669 hybrid_clock.cc:648] HybridClock initialized: now 1786021834179268 us; error 29 us; skew 500 ppm
Aug 06 13:10:34 dist-test-slave-q9h9 krb5kdc[11652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021834, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:34.186112 11669 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:10:34.186535 11669 webserver.cc:460] Webserver started at http://127.0.0.1:33767/ using document root <none> and password file <none>
I20260806 13:10:34.186718 11669 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:34.186765 11669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:34.186853 11669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:34.187762 11669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "3adb1e8c96404abba1fcd6873f087714"
format_stamp: "Formatted at 2026-08-06 13:10:34 on dist-test-slave-q9h9"
server_key: "1dd1b745ba16f96bc9658ca3fdcca2f2"
server_key_iv: "32490491b8282355129297931f2db7aa"
server_key_version: "encryptionkey@0"
I20260806 13:10:34.188037 11669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "3adb1e8c96404abba1fcd6873f087714"
format_stamp: "Formatted at 2026-08-06 13:10:34 on dist-test-slave-q9h9"
server_key: "1dd1b745ba16f96bc9658ca3fdcca2f2"
server_key_iv: "32490491b8282355129297931f2db7aa"
server_key_version: "encryptionkey@0"
I20260806 13:10:34.189167 11669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:10:34.189766 11690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:34.189917 11669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:10:34.189998 11669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "3adb1e8c96404abba1fcd6873f087714"
format_stamp: "Formatted at 2026-08-06 13:10:34 on dist-test-slave-q9h9"
server_key: "1dd1b745ba16f96bc9658ca3fdcca2f2"
server_key_iv: "32490491b8282355129297931f2db7aa"
server_key_version: "encryptionkey@0"
I20260806 13:10:34.190088 11669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:34.200717 11669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:34.202219 11669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:34.202320 11669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:34.205842 11669 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42963
I20260806 13:10:34.205874 11771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42963 every 8 connection(s)
I20260806 13:10:34.206179 11669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:34.206956 11773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:34.208935 11773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714: Bootstrap starting.
I20260806 13:10:34.209467 11773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:34.209719 11773 log.cc:824] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:34.210376 11773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714: No bootstrap required, opened a new log
I20260806 13:10:34.211838 11773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3adb1e8c96404abba1fcd6873f087714" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42963 } }
I20260806 13:10:34.211959 11773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:34.211990 11773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3adb1e8c96404abba1fcd6873f087714, State: Initialized, Role: FOLLOWER
I20260806 13:10:34.212116 11773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3adb1e8c96404abba1fcd6873f087714" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42963 } }
I20260806 13:10:34.212190 11773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:34.212222 11773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:34.212277 11773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:34.212743 11773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3adb1e8c96404abba1fcd6873f087714" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42963 } }
I20260806 13:10:34.212836 11773 leader_election.cc:302] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3adb1e8c96404abba1fcd6873f087714; no voters:
I20260806 13:10:34.212982 11773 leader_election.cc:288] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:34.213006 11776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:34.213125 11776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [term 1 LEADER]: Becoming Leader. State: Replica: 3adb1e8c96404abba1fcd6873f087714, State: Running, Role: LEADER
I20260806 13:10:34.213220 11773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:34.213264 11776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3adb1e8c96404abba1fcd6873f087714" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42963 } }
I20260806 13:10:34.213621 11778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3adb1e8c96404abba1fcd6873f087714. Latest consensus state: current_term: 1 leader_uuid: "3adb1e8c96404abba1fcd6873f087714" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3adb1e8c96404abba1fcd6873f087714" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42963 } } }
I20260806 13:10:34.213696 11778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:34.213728 11777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3adb1e8c96404abba1fcd6873f087714" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3adb1e8c96404abba1fcd6873f087714" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42963 } } }
I20260806 13:10:34.213811 11777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:34.214231 11785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:34.214673 11785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:34.214757 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 11669
I20260806 13:10:34.214879 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:34.215058 14209 external_mini_cluster.cc:1428] Setting key 37fb9d6f903cd341e34fa689d7e688d8
I20260806 13:10:34.216233 11785 catalog_manager.cc:1347] Generated new cluster ID: 722a57c07b7a4daba3893368f711f647
I20260806 13:10:34.216287 11785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:34.220742 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:34.215521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60741 (local address 127.0.0.1:42963)
0806 13:10:34.215764 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:34.215773 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:34.215806 (+ 33us) server_negotiation.cc:407] Connection header received
0806 13:10:34.215922 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:34.215928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:34.215967 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:34.216075 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:34.216316 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:34.217651 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:34.218809 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:34.219411 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:34.219504 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:34.220131 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:34.220134 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:34.220134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:34.220320 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:34.220353 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:34.220367 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:34.220442 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:34.220544 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 13:10:34.220566 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":72,"threads_started":1}
I20260806 13:10:34.221549 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:34.226155 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:34.222947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53219 (local address 127.0.0.1:42963)
0806 13:10:34.222975 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:34.222976 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:34.223100 (+ 124us) server_negotiation.cc:407] Connection header received
0806 13:10:34.223154 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:34.223154 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:34.223162 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:34.223215 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:34.223335 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:34.224022 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:34.225002 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:34.225299 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:34.225349 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:34.225905 (+ 556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:34.225906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:34.225907 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:34.225910 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:34.225922 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:34.225926 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:34.225972 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:34.226061 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 13:10:34.226071 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 13:10:34.236128 11785 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:34.236552 11785 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:34.240383 11785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3adb1e8c96404abba1fcd6873f087714: Generated new TSK 0
I20260806 13:10:34.240541 11785 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:34.244534 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 11669
W20260806 13:10:34.260519 11648 connection.cc:535] client connection to 127.0.0.1:42963 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42963): Connection reset by peer (error 104)
2026-08-06T13:10:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:34 dist-test-slave-q9h9 krb5kdc[11818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:34 dist-test-slave-q9h9 krb5kdc[11818](info): set up 2 sockets
Aug 06 13:10:34 dist-test-slave-q9h9 krb5kdc[11818](info): commencing operation
krb5kdc: starting...
W20260806 13:10:35.382613 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:35 dist-test-slave-q9h9 krb5kdc[11818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:10:35.488140 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33203
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:35.565656 11836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:35.565833 11836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:35.565863 11836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:35.566957 11836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:35.566998 11836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:35.567018 11836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:35.567034 11836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:35.567049 11836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:35.568495 11836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33203
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:35.568723 11836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:35.568910 11836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:35.573280 11842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:35.573251 11845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:35.573269 11843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:35.573437 11836 server_base.cc:1034] running on GCE node
I20260806 13:10:35.573697 11836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:35.573937 11836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:35.575069 11836 hybrid_clock.cc:648] HybridClock initialized: now 1786021835575063 us; error 40 us; skew 500 ppm
Aug 06 13:10:35 dist-test-slave-q9h9 krb5kdc[11818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021835, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:35.581322 11836 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:10:35.581746 11836 webserver.cc:460] Webserver started at http://127.0.0.1:34345/ using document root <none> and password file <none>
I20260806 13:10:35.581928 11836 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:35.581974 11836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:35.582120 11836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:35.583036 11836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "315eaa18dff94738b9bbd97960ec6ee5"
format_stamp: "Formatted at 2026-08-06 13:10:35 on dist-test-slave-q9h9"
server_key: "ec0355bf0460e2c24f48a4c2431ac968"
server_key_iv: "3add740f16518b2d0bb33e2fca033fb6"
server_key_version: "encryptionkey@0"
I20260806 13:10:35.583344 11836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "315eaa18dff94738b9bbd97960ec6ee5"
format_stamp: "Formatted at 2026-08-06 13:10:35 on dist-test-slave-q9h9"
server_key: "ec0355bf0460e2c24f48a4c2431ac968"
server_key_iv: "3add740f16518b2d0bb33e2fca033fb6"
server_key_version: "encryptionkey@0"
I20260806 13:10:35.584492 11836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:10:35.585106 11855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:35.585256 11836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:10:35.585335 11836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "315eaa18dff94738b9bbd97960ec6ee5"
format_stamp: "Formatted at 2026-08-06 13:10:35 on dist-test-slave-q9h9"
server_key: "ec0355bf0460e2c24f48a4c2431ac968"
server_key_iv: "3add740f16518b2d0bb33e2fca033fb6"
server_key_version: "encryptionkey@0"
I20260806 13:10:35.585395 11836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:35.588431 11836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:35.589960 11836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:35.590098 11836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:35.593595 11836 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33203
I20260806 13:10:35.593624 11937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33203 every 8 connection(s)
I20260806 13:10:35.593899 11836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:35.594811 11938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:35.596777 11938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5: Bootstrap starting.
I20260806 13:10:35.597296 11938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:35.597536 11938 log.cc:824] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:35.598156 11938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5: No bootstrap required, opened a new log
I20260806 13:10:35.599623 11938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "315eaa18dff94738b9bbd97960ec6ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33203 } }
I20260806 13:10:35.599743 11938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:35.599778 11938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315eaa18dff94738b9bbd97960ec6ee5, State: Initialized, Role: FOLLOWER
I20260806 13:10:35.599906 11938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "315eaa18dff94738b9bbd97960ec6ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33203 } }
I20260806 13:10:35.599964 11938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:35.599984 11938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:35.600028 11938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:35.600502 11938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "315eaa18dff94738b9bbd97960ec6ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33203 } }
I20260806 13:10:35.600591 11938 leader_election.cc:302] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 315eaa18dff94738b9bbd97960ec6ee5; no voters:
I20260806 13:10:35.600736 11938 leader_election.cc:288] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:35.600773 11945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:35.600907 11945 raft_consensus.cc:686] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [term 1 LEADER]: Becoming Leader. State: Replica: 315eaa18dff94738b9bbd97960ec6ee5, State: Running, Role: LEADER
I20260806 13:10:35.600951 11938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:35.601020 11945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "315eaa18dff94738b9bbd97960ec6ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33203 } }
I20260806 13:10:35.601373 11947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 315eaa18dff94738b9bbd97960ec6ee5. Latest consensus state: current_term: 1 leader_uuid: "315eaa18dff94738b9bbd97960ec6ee5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "315eaa18dff94738b9bbd97960ec6ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33203 } } }
I20260806 13:10:35.601356 11946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "315eaa18dff94738b9bbd97960ec6ee5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "315eaa18dff94738b9bbd97960ec6ee5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33203 } } }
I20260806 13:10:35.601509 11947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:35.601595 11946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:35.601890 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 11836
I20260806 13:10:35.601986 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:35.602164 14209 external_mini_cluster.cc:1428] Setting key c6297f952e4ac8e865628ee86930e342
I20260806 13:10:35.602510 11951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:35.603194 11951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:35.604533 11961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:35.602738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42723 (local address 127.0.0.1:33203)
0806 13:10:35.602968 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:35.602977 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:35.603008 (+ 31us) server_negotiation.cc:407] Connection header received
0806 13:10:35.603114 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:35.603121 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:35.603149 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:35.603302 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:35.603946 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:35.603950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:35.603950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:35.603954 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:35.603990 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:35.604002 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:35.604297 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:35.604328 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 13:10:35.604365 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":112,"threads_started":1}
I20260806 13:10:35.604611 11951 catalog_manager.cc:1347] Generated new cluster ID: 7f4d6925c8884c6383f1720ff5162460
I20260806 13:10:35.604727 11951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:35.605459 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:35.611215 11961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:35.610232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60301 (local address 127.0.0.1:33203)
0806 13:10:35.610263 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:35.610265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:35.610450 (+ 185us) server_negotiation.cc:407] Connection header received
0806 13:10:35.610505 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:35.610506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:35.610521 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:35.610564 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:35.611020 (+ 456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:35.611022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:35.611022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:35.611026 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:35.611038 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:35.611042 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:35.611086 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:35.611135 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 13:10:35.611146 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:10:35.628841 11951 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:35.628986 11951 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:35.634092 11951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 315eaa18dff94738b9bbd97960ec6ee5: Generated new TSK 0
I20260806 13:10:35.634260 11951 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:35.643713 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 11836
2026-08-06T13:10:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1394 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:35 dist-test-slave-q9h9 krb5kdc[11985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:35 dist-test-slave-q9h9 krb5kdc[11985](info): set up 2 sockets
Aug 06 13:10:35 dist-test-slave-q9h9 krb5kdc[11985](info): commencing operation
krb5kdc: starting...
W20260806 13:10:36.758920 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:36 dist-test-slave-q9h9 krb5kdc[11985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:10:36.866639 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36635
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:36.944275 12001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:36.944450 12001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:36.944471 12001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:36.945567 12001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:36.945606 12001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:36.945619 12001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:36.945637 12001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:36.945660 12001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:36.947108 12001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:36.947325 12001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:36.947515 12001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:36.951861 12009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:36.951860 12013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:36.952020 12001 server_base.cc:1034] running on GCE node
W20260806 13:10:36.951886 12010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:36.952373 12001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:36.952656 12001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:36.953795 12001 hybrid_clock.cc:648] HybridClock initialized: now 1786021836953785 us; error 28 us; skew 500 ppm
Aug 06 13:10:36 dist-test-slave-q9h9 krb5kdc[11985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021836, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:36.955992 12001 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:10:36.956378 12001 webserver.cc:460] Webserver started at http://127.0.0.1:45697/ using document root <none> and password file <none>
I20260806 13:10:36.956562 12001 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:36.956609 12001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:36.956724 12001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:36.957653 12001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "d46badba3aab4da1a93196abade0f389"
format_stamp: "Formatted at 2026-08-06 13:10:36 on dist-test-slave-q9h9"
server_key: "22ec47351d2428857c8f7999fb0d3480"
server_key_iv: "51f05fb97463e9b628c9daa1a879b38b"
server_key_version: "encryptionkey@0"
I20260806 13:10:36.957963 12001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "d46badba3aab4da1a93196abade0f389"
format_stamp: "Formatted at 2026-08-06 13:10:36 on dist-test-slave-q9h9"
server_key: "22ec47351d2428857c8f7999fb0d3480"
server_key_iv: "51f05fb97463e9b628c9daa1a879b38b"
server_key_version: "encryptionkey@0"
I20260806 13:10:36.959148 12001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:10:36.959782 12020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:36.959928 12001 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:36.960007 12001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "d46badba3aab4da1a93196abade0f389"
format_stamp: "Formatted at 2026-08-06 13:10:36 on dist-test-slave-q9h9"
server_key: "22ec47351d2428857c8f7999fb0d3480"
server_key_iv: "51f05fb97463e9b628c9daa1a879b38b"
server_key_version: "encryptionkey@0"
I20260806 13:10:36.960067 12001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:36.969331 12001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:36.970966 12001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:36.971086 12001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:36.974630 12001 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36635
I20260806 13:10:36.974665 12102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36635 every 8 connection(s)
I20260806 13:10:36.974957 12001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:36.975733 12103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:36.977715 12103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389: Bootstrap starting.
I20260806 13:10:36.978266 12103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:36.978514 12103 log.cc:824] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:36.979135 12103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389: No bootstrap required, opened a new log
I20260806 13:10:36.980535 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 12001
I20260806 13:10:36.980631 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:36.980651 12103 raft_consensus.cc:348] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46badba3aab4da1a93196abade0f389" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36635 } }
I20260806 13:10:36.980785 14209 external_mini_cluster.cc:1428] Setting key 08c66d1f370e02af56a553b3d1271eaa
I20260806 13:10:36.980795 12103 raft_consensus.cc:374] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:36.980822 12103 raft_consensus.cc:729] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d46badba3aab4da1a93196abade0f389, State: Initialized, Role: FOLLOWER
I20260806 13:10:36.980955 12103 consensus_queue.cc:260] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46badba3aab4da1a93196abade0f389" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36635 } }
I20260806 13:10:36.981034 12103 raft_consensus.cc:388] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:36.981065 12103 raft_consensus.cc:482] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:36.981161 12103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:36.981662 12103 raft_consensus.cc:504] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46badba3aab4da1a93196abade0f389" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36635 } }
I20260806 13:10:36.981748 12103 leader_election.cc:302] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d46badba3aab4da1a93196abade0f389; no voters:
I20260806 13:10:36.981889 12103 leader_election.cc:288] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:36.981932 12110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:36.982097 12110 raft_consensus.cc:686] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [term 1 LEADER]: Becoming Leader. State: Replica: d46badba3aab4da1a93196abade0f389, State: Running, Role: LEADER
I20260806 13:10:36.982163 12103 sys_catalog.cc:564] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:36.982191 12110 consensus_queue.cc:237] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46badba3aab4da1a93196abade0f389" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36635 } }
I20260806 13:10:36.982659 12112 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d46badba3aab4da1a93196abade0f389. Latest consensus state: current_term: 1 leader_uuid: "d46badba3aab4da1a93196abade0f389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46badba3aab4da1a93196abade0f389" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36635 } } }
I20260806 13:10:36.982802 12112 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:36.982995 12111 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d46badba3aab4da1a93196abade0f389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46badba3aab4da1a93196abade0f389" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36635 } } }
I20260806 13:10:36.983114 12111 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:36.983264 12122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:36.983830 12122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:36.985112 12122 catalog_manager.cc:1347] Generated new cluster ID: d60f7d0710aa4487a307823f6ad60e6e
I20260806 13:10:36.985175 12122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:10:36 dist-test-slave-q9h9 krb5kdc[11985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 13:10:36.989609 12122 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:36.990263 12122 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:36.990667 12109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:36.981153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43413 (local address 127.0.0.1:36635)
0806 13:10:36.981322 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:36.981332 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:36.981368 (+ 36us) server_negotiation.cc:407] Connection header received
0806 13:10:36.981660 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:36.981666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:36.981705 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:36.981843 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:36.982669 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:36.983988 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:36.985082 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:36.985526 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:36.985621 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:36.987740 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:36.987753 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:36.987753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:36.987764 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:36.989200 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:36.989549 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:36.989552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:36.989552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:36.989629 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:36.989836 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:36.989837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:36.989837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:36.990065 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:36.990211 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:36.990279 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 13:10:36.990523 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260806 13:10:36.991413 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:36.999190 12109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:36.992723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59711 (local address 127.0.0.1:36635)
0806 13:10:36.992752 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:36.992753 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:36.993257 (+ 504us) server_negotiation.cc:407] Connection header received
0806 13:10:36.993343 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:36.993344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:36.993354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:36.993380 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:36.994052 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:36.994701 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:36.995711 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:36.995963 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:36.995996 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:36.997017 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:36.997018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:36.997018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:36.997025 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:36.998267 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:36.998552 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:36.998553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:36.998554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:36.998603 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:36.998765 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:36.998766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:36.998766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:36.998863 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:36.998939 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:36.999041 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 13:10:36.999113 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 13:10:37.000280 12122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d46badba3aab4da1a93196abade0f389: Generated new TSK 0
I20260806 13:10:37.000429 12122 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:37.002271 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 12001
2026-08-06T13:10:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1358 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:37 dist-test-slave-q9h9 krb5kdc[12148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:37 dist-test-slave-q9h9 krb5kdc[12148](info): set up 2 sockets
Aug 06 13:10:37 dist-test-slave-q9h9 krb5kdc[12148](info): commencing operation
krb5kdc: starting...
W20260806 13:10:38.115645 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:38 dist-test-slave-q9h9 krb5kdc[12148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:10:38.222936 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43843
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:38.301918 12166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:38.302142 12166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:38.302186 12166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:38.303299 12166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:38.303339 12166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:38.303359 12166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:38.303373 12166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:38.303390 12166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:38.304819 12166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:38.305045 12166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:38.305248 12166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:38.309691 12173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:38.309705 12175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:38.309733 12172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:38.309835 12166 server_base.cc:1034] running on GCE node
I20260806 13:10:38.310165 12166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:38.310441 12166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:38.311585 12166 hybrid_clock.cc:648] HybridClock initialized: now 1786021838311570 us; error 28 us; skew 500 ppm
Aug 06 13:10:38 dist-test-slave-q9h9 krb5kdc[12148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:38.313850 12166 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:10:38.314301 12166 webserver.cc:460] Webserver started at http://127.0.0.1:35193/ using document root <none> and password file <none>
I20260806 13:10:38.314520 12166 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:38.314566 12166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:38.314687 12166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:38.315627 12166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "1f0136ba63d14be4be55b15493582e99"
format_stamp: "Formatted at 2026-08-06 13:10:38 on dist-test-slave-q9h9"
server_key: "dfefbdbcbad75f273a6d17ed29daa1ba"
server_key_iv: "51ae518d0499f6e62cbc947c218d7b1f"
server_key_version: "encryptionkey@0"
I20260806 13:10:38.315943 12166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "1f0136ba63d14be4be55b15493582e99"
format_stamp: "Formatted at 2026-08-06 13:10:38 on dist-test-slave-q9h9"
server_key: "dfefbdbcbad75f273a6d17ed29daa1ba"
server_key_iv: "51ae518d0499f6e62cbc947c218d7b1f"
server_key_version: "encryptionkey@0"
I20260806 13:10:38.317148 12166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 13:10:38.317766 12184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:38.317914 12166 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:38.317994 12166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "1f0136ba63d14be4be55b15493582e99"
format_stamp: "Formatted at 2026-08-06 13:10:38 on dist-test-slave-q9h9"
server_key: "dfefbdbcbad75f273a6d17ed29daa1ba"
server_key_iv: "51ae518d0499f6e62cbc947c218d7b1f"
server_key_version: "encryptionkey@0"
I20260806 13:10:38.318080 12166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:38.328919 12166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:38.330825 12166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:38.330952 12166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:38.334601 12166 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43843
I20260806 13:10:38.334640 12266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43843 every 8 connection(s)
I20260806 13:10:38.334931 12166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:38.335711 12267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:38.336755 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 12166
I20260806 13:10:38.336876 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:38.337073 14209 external_mini_cluster.cc:1428] Setting key f5c5979690fd750d10473dc703f08b90
I20260806 13:10:38.338101 12267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99: Bootstrap starting.
I20260806 13:10:38.338712 12267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:38.338967 12267 log.cc:824] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:38.339568 12267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99: No bootstrap required, opened a new log
I20260806 13:10:38.341638 12267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f0136ba63d14be4be55b15493582e99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43843 } }
I20260806 13:10:38.341760 12267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:38.341780 12267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0136ba63d14be4be55b15493582e99, State: Initialized, Role: FOLLOWER
I20260806 13:10:38.341897 12267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f0136ba63d14be4be55b15493582e99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43843 } }
I20260806 13:10:38.341959 12267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:38.341988 12267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:38.342083 12267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:38.342283 12273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:38.337542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60587 (local address 127.0.0.1:43843)
0806 13:10:38.337705 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:38.337713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:38.337736 (+ 23us) server_negotiation.cc:407] Connection header received
0806 13:10:38.338162 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:38.338172 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:38.338231 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:38.338345 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:38.338578 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:38.339485 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:38.340530 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:38.340938 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:38.341012 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:38.341600 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:38.341603 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:38.341603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:38.341754 (+ 151us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:38.341789 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:38.341804 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:38.341867 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:38.341941 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 13:10:38.342108 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260806 13:10:38.342653 12267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f0136ba63d14be4be55b15493582e99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43843 } }
I20260806 13:10:38.342751 12267 leader_election.cc:302] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f0136ba63d14be4be55b15493582e99; no voters:
I20260806 13:10:38.342902 12267 leader_election.cc:288] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:38.342958 12275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:38.343070 12275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [term 1 LEADER]: Becoming Leader. State: Replica: 1f0136ba63d14be4be55b15493582e99, State: Running, Role: LEADER
I20260806 13:10:38.343115 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:38.343147 12267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:38.343204 12275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f0136ba63d14be4be55b15493582e99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43843 } }
I20260806 13:10:38.343503 12275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f0136ba63d14be4be55b15493582e99. Latest consensus state: current_term: 1 leader_uuid: "1f0136ba63d14be4be55b15493582e99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f0136ba63d14be4be55b15493582e99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43843 } } }
I20260806 13:10:38.343556 12277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f0136ba63d14be4be55b15493582e99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f0136ba63d14be4be55b15493582e99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43843 } } }
I20260806 13:10:38.343569 12275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:38.343653 12277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:38.343811 12283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:38.344444 12283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:38.345969 12283 catalog_manager.cc:1347] Generated new cluster ID: a87559edccda410480dfed099d364b04
I20260806 13:10:38.346038 12283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:38.347760 12273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:38.344656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60223 (local address 127.0.0.1:43843)
0806 13:10:38.344700 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:38.344701 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:38.344938 (+ 237us) server_negotiation.cc:407] Connection header received
0806 13:10:38.344982 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:38.344983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:38.344992 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:38.345045 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:38.345206 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:38.345712 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:38.346673 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:38.346951 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:38.347020 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:38.347543 (+ 523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:38.347544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:38.347545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:38.347549 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:38.347562 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:38.347565 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:38.347621 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:38.347672 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 13:10:38.347692 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 13:10:38.394575 12283 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:38.395078 12283 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:38.397876 12283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f0136ba63d14be4be55b15493582e99: Generated new TSK 0
I20260806 13:10:38.398047 12283 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:38.400435 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 12166
2026-08-06T13:10:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:38 dist-test-slave-q9h9 krb5kdc[12311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:38 dist-test-slave-q9h9 krb5kdc[12311](info): set up 2 sockets
Aug 06 13:10:38 dist-test-slave-q9h9 krb5kdc[12311](info): commencing operation
krb5kdc: starting...
W20260806 13:10:39.502995 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:39 dist-test-slave-q9h9 krb5kdc[12311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:10:39.611783 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:39.693082 12328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:39.693276 12328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:39.693307 12328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:39.694432 12328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:39.694473 12328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:39.694490 12328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:39.694506 12328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:39.694522 12328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:39.695986 12328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:39.696228 12328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:39.696421 12328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:39.700769 12333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:39.700776 12339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:39.700961 12328 server_base.cc:1034] running on GCE node
W20260806 13:10:39.700779 12335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:39.701259 12328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:39.701517 12328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:39.702653 12328 hybrid_clock.cc:648] HybridClock initialized: now 1786021839702623 us; error 48 us; skew 500 ppm
Aug 06 13:10:39 dist-test-slave-q9h9 krb5kdc[12311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021839, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:39.708673 12328 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:10:39.709030 12328 webserver.cc:460] Webserver started at http://127.0.0.1:33499/ using document root <none> and password file <none>
I20260806 13:10:39.709226 12328 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:39.709264 12328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:39.709349 12328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:39.710294 12328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "910472d76bed4ddebd660bc313e5098a"
format_stamp: "Formatted at 2026-08-06 13:10:39 on dist-test-slave-q9h9"
server_key: "4966d6a6d0f632b0cb70fa018cc77259"
server_key_iv: "fbb0470a8827e07ddb83d213475e922c"
server_key_version: "encryptionkey@0"
I20260806 13:10:39.710566 12328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "910472d76bed4ddebd660bc313e5098a"
format_stamp: "Formatted at 2026-08-06 13:10:39 on dist-test-slave-q9h9"
server_key: "4966d6a6d0f632b0cb70fa018cc77259"
server_key_iv: "fbb0470a8827e07ddb83d213475e922c"
server_key_version: "encryptionkey@0"
I20260806 13:10:39.711679 12328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:10:39.712291 12350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:39.712433 12328 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:39.712508 12328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "910472d76bed4ddebd660bc313e5098a"
format_stamp: "Formatted at 2026-08-06 13:10:39 on dist-test-slave-q9h9"
server_key: "4966d6a6d0f632b0cb70fa018cc77259"
server_key_iv: "fbb0470a8827e07ddb83d213475e922c"
server_key_version: "encryptionkey@0"
I20260806 13:10:39.712568 12328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:39.715680 12328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:39.717170 12328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:39.717291 12328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:39.720808 12328 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43967
I20260806 13:10:39.720856 12436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43967 every 8 connection(s)
I20260806 13:10:39.721102 12328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:39.721984 12437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:39.723966 12437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a: Bootstrap starting.
I20260806 13:10:39.724484 12437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:39.724731 12437 log.cc:824] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:39.725329 12437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a: No bootstrap required, opened a new log
I20260806 13:10:39.725669 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 12328
I20260806 13:10:39.725783 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:39.725965 14209 external_mini_cluster.cc:1428] Setting key 634cfc8cfadc189ae15ad02ba6ed5873
I20260806 13:10:39.727437 12437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "910472d76bed4ddebd660bc313e5098a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } }
I20260806 13:10:39.727579 12437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:39.727625 12437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 910472d76bed4ddebd660bc313e5098a, State: Initialized, Role: FOLLOWER
I20260806 13:10:39.727749 12437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "910472d76bed4ddebd660bc313e5098a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } }
I20260806 13:10:39.727839 12437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:39.727870 12437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:39.727963 12437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:39.728147 12445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:39.726422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34447 (local address 127.0.0.1:43967)
0806 13:10:39.726575 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:39.726582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:39.726603 (+ 21us) server_negotiation.cc:407] Connection header received
0806 13:10:39.727032 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:39.727038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:39.727062 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:39.727163 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:39.727735 (+ 572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:39.727739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:39.727739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:39.727743 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:39.727779 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:39.727788 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:39.727847 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:39.727940 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 13:10:39.727980 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260806 13:10:39.728854 12437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "910472d76bed4ddebd660bc313e5098a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } }
I20260806 13:10:39.728952 12437 leader_election.cc:302] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 910472d76bed4ddebd660bc313e5098a; no voters:
I20260806 13:10:39.728976 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:39.729104 12437 leader_election.cc:288] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:39.729146 12446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:39.729288 12446 raft_consensus.cc:686] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [term 1 LEADER]: Becoming Leader. State: Replica: 910472d76bed4ddebd660bc313e5098a, State: Running, Role: LEADER
I20260806 13:10:39.729342 12437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:39.730116 12446 consensus_queue.cc:237] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "910472d76bed4ddebd660bc313e5098a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } }
W20260806 13:10:39.730954 12478 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 13:10:39.730798 12448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "910472d76bed4ddebd660bc313e5098a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "910472d76bed4ddebd660bc313e5098a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } } }
I20260806 13:10:39.731071 12448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:39.730659 12447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 910472d76bed4ddebd660bc313e5098a. Latest consensus state: current_term: 1 leader_uuid: "910472d76bed4ddebd660bc313e5098a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "910472d76bed4ddebd660bc313e5098a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43967 } } }
I20260806 13:10:39.731217 12447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a [sys.catalog]: This master's current role is: LEADER
W20260806 13:10:39.731290 12478 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:10:39.731370 12483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:39.731556 12483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:39.732394 12445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:39.730406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57641 (local address 127.0.0.1:43967)
0806 13:10:39.730443 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:39.730445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:39.731559 (+ 1114us) server_negotiation.cc:407] Connection header received
0806 13:10:39.731633 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:39.731634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:39.731638 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:39.731685 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:39.732189 (+ 504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:39.732191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:39.732191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:39.732196 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:39.732208 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:39.732212 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:39.732238 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:39.732331 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 13:10:39.732343 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:10:39.732916 12483 catalog_manager.cc:1347] Generated new cluster ID: c17f422c9bc44e51a35d96940cc26f0c
I20260806 13:10:39.732959 12483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:39.739652 12483 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:39.739796 12483 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:39.746572 12483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 910472d76bed4ddebd660bc313e5098a: Generated new TSK 0
I20260806 13:10:39.746716 12483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:39.751391 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 12328
2026-08-06T13:10:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1357 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:39 dist-test-slave-q9h9 krb5kdc[12488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:39 dist-test-slave-q9h9 krb5kdc[12488](info): set up 2 sockets
Aug 06 13:10:39 dist-test-slave-q9h9 krb5kdc[12488](info): commencing operation
krb5kdc: starting...
W20260806 13:10:40.862813 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:40 dist-test-slave-q9h9 krb5kdc[12488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:10:40.969975 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46159
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:41.049000 12506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:41.049176 12506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:41.049206 12506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:41.050309 12506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:41.050347 12506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:41.050366 12506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:41.050381 12506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:41.050397 12506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:41.051802 12506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:41.052032 12506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:41.052225 12506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:41.056713 12515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:41.056705 12516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:41.056797 12506 server_base.cc:1034] running on GCE node
W20260806 13:10:41.056704 12518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:41.057099 12506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:41.057369 12506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:41.058502 12506 hybrid_clock.cc:648] HybridClock initialized: now 1786021841058495 us; error 28 us; skew 500 ppm
Aug 06 13:10:41 dist-test-slave-q9h9 krb5kdc[12488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021841, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:41.060835 12506 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:10:41.061245 12506 webserver.cc:460] Webserver started at http://127.0.0.1:43455/ using document root <none> and password file <none>
I20260806 13:10:41.061451 12506 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:41.061493 12506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:41.061615 12506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:41.062573 12506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "dbfabec9cef84f3d9411cac33e9d30dd"
format_stamp: "Formatted at 2026-08-06 13:10:41 on dist-test-slave-q9h9"
server_key: "107417d36cddf39401435bb5a01b91f6"
server_key_iv: "ea355936f6f54f01dc844d5fe5cda6d8"
server_key_version: "encryptionkey@0"
I20260806 13:10:41.062873 12506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "dbfabec9cef84f3d9411cac33e9d30dd"
format_stamp: "Formatted at 2026-08-06 13:10:41 on dist-test-slave-q9h9"
server_key: "107417d36cddf39401435bb5a01b91f6"
server_key_iv: "ea355936f6f54f01dc844d5fe5cda6d8"
server_key_version: "encryptionkey@0"
I20260806 13:10:41.064018 12506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:41.064652 12525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:41.064793 12506 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:41.064873 12506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "dbfabec9cef84f3d9411cac33e9d30dd"
format_stamp: "Formatted at 2026-08-06 13:10:41 on dist-test-slave-q9h9"
server_key: "107417d36cddf39401435bb5a01b91f6"
server_key_iv: "ea355936f6f54f01dc844d5fe5cda6d8"
server_key_version: "encryptionkey@0"
I20260806 13:10:41.064930 12506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:41.080268 12506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:41.081750 12506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:41.081872 12506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:41.085534 12506 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46159
I20260806 13:10:41.085566 12606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46159 every 8 connection(s)
I20260806 13:10:41.085848 12506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:41.086627 12610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:41.088624 12610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd: Bootstrap starting.
I20260806 13:10:41.089136 12610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:41.089393 12610 log.cc:824] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:41.090000 12610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd: No bootstrap required, opened a new log
I20260806 13:10:41.091490 12610 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbfabec9cef84f3d9411cac33e9d30dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46159 } }
I20260806 13:10:41.091614 12610 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:41.091645 12610 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbfabec9cef84f3d9411cac33e9d30dd, State: Initialized, Role: FOLLOWER
I20260806 13:10:41.091773 12610 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbfabec9cef84f3d9411cac33e9d30dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46159 } }
I20260806 13:10:41.091833 12610 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:41.091854 12610 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:41.091898 12610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:41.092386 12610 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbfabec9cef84f3d9411cac33e9d30dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46159 } }
I20260806 13:10:41.092469 12610 leader_election.cc:302] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbfabec9cef84f3d9411cac33e9d30dd; no voters:
I20260806 13:10:41.092609 12610 leader_election.cc:288] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:41.092657 12615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:41.092834 12610 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:41.092849 12615 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [term 1 LEADER]: Becoming Leader. State: Replica: dbfabec9cef84f3d9411cac33e9d30dd, State: Running, Role: LEADER
I20260806 13:10:41.093009 12615 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbfabec9cef84f3d9411cac33e9d30dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46159 } }
I20260806 13:10:41.093390 12617 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbfabec9cef84f3d9411cac33e9d30dd. Latest consensus state: current_term: 1 leader_uuid: "dbfabec9cef84f3d9411cac33e9d30dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbfabec9cef84f3d9411cac33e9d30dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46159 } } }
I20260806 13:10:41.093420 12616 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbfabec9cef84f3d9411cac33e9d30dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbfabec9cef84f3d9411cac33e9d30dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46159 } } }
I20260806 13:10:41.093544 12616 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:41.093529 12617 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:41.093973 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 12506
I20260806 13:10:41.094089 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:41.094233 14209 external_mini_cluster.cc:1428] Setting key 3a5e3df946f7d9be2b69719f8a31bbdc
I20260806 13:10:41.094560 12621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:41.095045 12621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:41.096704 12621 catalog_manager.cc:1347] Generated new cluster ID: 5df260516ae14d17b4daedf096060ae1
I20260806 13:10:41.096762 12621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:10:41 dist-test-slave-q9h9 krb5kdc[12488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 13:10:41.105257 12637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:41.094985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:39021 (local address 127.0.0.1:46159)
0806 13:10:41.095189 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:41.095198 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:41.095230 (+ 32us) server_negotiation.cc:407] Connection header received
0806 13:10:41.095337 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:41.095343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:41.095396 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:41.095531 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:41.096194 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:41.097406 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:41.098488 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:41.099078 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:41.099167 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:41.101785 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:41.101790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:41.101790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:41.101817 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:41.103668 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:41.104071 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:41.104073 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:41.104074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:41.104197 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:41.104396 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:41.104397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:41.104398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:41.104691 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:41.104877 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:41.104957 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 13:10:41.105057 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":63,"threads_started":1}
I20260806 13:10:41.106187 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:41.114154 12637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:41.107633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:51817 (local address 127.0.0.1:46159)
0806 13:10:41.107662 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:41.107664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:41.107753 (+ 89us) server_negotiation.cc:407] Connection header received
0806 13:10:41.107809 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:41.107810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:41.107817 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:41.107869 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:41.108418 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:41.109118 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:41.110110 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:41.110424 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:41.110475 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:41.111685 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:41.111686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:41.111686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:41.111692 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:41.113057 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:41.113442 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:41.113443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:41.113443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:41.113491 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:41.113677 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:41.113678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:41.113678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:41.113780 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:41.113892 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:41.113987 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 13:10:41.114076 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 13:10:41.131487 12621 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:41.131955 12621 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:41.134128 12621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbfabec9cef84f3d9411cac33e9d30dd: Generated new TSK 0
I20260806 13:10:41.134287 12621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:41.138062 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 12506
W20260806 13:10:41.152699 12486 connection.cc:535] client connection to 127.0.0.1:46159 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46159): Connection reset by peer (error 104)
2026-08-06T13:10:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1383 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:41 dist-test-slave-q9h9 krb5kdc[12653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:41 dist-test-slave-q9h9 krb5kdc[12653](info): set up 2 sockets
Aug 06 13:10:41 dist-test-slave-q9h9 krb5kdc[12653](info): commencing operation
krb5kdc: starting...
W20260806 13:10:42.252336 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:42 dist-test-slave-q9h9 krb5kdc[12653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:10:42.358605 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41779
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:42.435879 12671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:42.436049 12671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:42.436069 12671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:42.437129 12671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:42.437160 12671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:42.437172 12671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:42.437184 12671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:42.437194 12671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:42.438617 12671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:42.438808 12671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:42.438971 12671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:42.443327 12679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:42.443362 12677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:42.443360 12682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:42.443392 12671 server_base.cc:1034] running on GCE node
I20260806 13:10:42.443712 12671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:42.443961 12671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:42.445143 12671 hybrid_clock.cc:648] HybridClock initialized: now 1786021842445128 us; error 30 us; skew 500 ppm
Aug 06 13:10:42 dist-test-slave-q9h9 krb5kdc[12653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:42.447440 12671 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:10:42.447822 12671 webserver.cc:460] Webserver started at http://127.0.0.1:34875/ using document root <none> and password file <none>
I20260806 13:10:42.448015 12671 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:42.448061 12671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:42.448187 12671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:42.449108 12671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "810c271a5c834a0b97a9735d96e12cd2"
format_stamp: "Formatted at 2026-08-06 13:10:42 on dist-test-slave-q9h9"
server_key: "80dd351e4faa98d238b54ab383fb04cb"
server_key_iv: "ea34a5860a1e53030ba48567d0d1bbf7"
server_key_version: "encryptionkey@0"
I20260806 13:10:42.449421 12671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "810c271a5c834a0b97a9735d96e12cd2"
format_stamp: "Formatted at 2026-08-06 13:10:42 on dist-test-slave-q9h9"
server_key: "80dd351e4faa98d238b54ab383fb04cb"
server_key_iv: "ea34a5860a1e53030ba48567d0d1bbf7"
server_key_version: "encryptionkey@0"
I20260806 13:10:42.450634 12671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 13:10:42.451272 12695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:42.451413 12671 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:42.451494 12671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "810c271a5c834a0b97a9735d96e12cd2"
format_stamp: "Formatted at 2026-08-06 13:10:42 on dist-test-slave-q9h9"
server_key: "80dd351e4faa98d238b54ab383fb04cb"
server_key_iv: "ea34a5860a1e53030ba48567d0d1bbf7"
server_key_version: "encryptionkey@0"
I20260806 13:10:42.451551 12671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:42.462379 12671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:42.463891 12671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:42.464012 12671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:42.467612 12671 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41779
I20260806 13:10:42.467651 12771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41779 every 8 connection(s)
I20260806 13:10:42.467935 12671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:42.468681 12772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:42.470659 12772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2: Bootstrap starting.
I20260806 13:10:42.471165 12772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:42.471392 12772 log.cc:824] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:42.471985 12772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2: No bootstrap required, opened a new log
I20260806 13:10:42.472502 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 12671
I20260806 13:10:42.472618 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:42.472798 14209 external_mini_cluster.cc:1428] Setting key aaf71f346580b2f8129f6099a9d12ee1
I20260806 13:10:42.473933 12772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810c271a5c834a0b97a9735d96e12cd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41779 } }
I20260806 13:10:42.474089 12772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:42.474115 12772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 810c271a5c834a0b97a9735d96e12cd2, State: Initialized, Role: FOLLOWER
I20260806 13:10:42.474226 12772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810c271a5c834a0b97a9735d96e12cd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41779 } }
I20260806 13:10:42.474298 12772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:42.474329 12772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:42.474421 12772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:42.474946 12772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810c271a5c834a0b97a9735d96e12cd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41779 } }
I20260806 13:10:42.475039 12772 leader_election.cc:302] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 810c271a5c834a0b97a9735d96e12cd2; no voters:
I20260806 13:10:42.475184 12772 leader_election.cc:288] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:42.475225 12777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:42.475382 12777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [term 1 LEADER]: Becoming Leader. State: Replica: 810c271a5c834a0b97a9735d96e12cd2, State: Running, Role: LEADER
I20260806 13:10:42.475440 12772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:42.475553 12777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810c271a5c834a0b97a9735d96e12cd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41779 } }
I20260806 13:10:42.475850 12779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "810c271a5c834a0b97a9735d96e12cd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810c271a5c834a0b97a9735d96e12cd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41779 } } }
I20260806 13:10:42.475854 12778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 810c271a5c834a0b97a9735d96e12cd2. Latest consensus state: current_term: 1 leader_uuid: "810c271a5c834a0b97a9735d96e12cd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "810c271a5c834a0b97a9735d96e12cd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41779 } } }
I20260806 13:10:42.475935 12779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:42.475939 12778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:42.476275 12787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:42.476706 12787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:42.478390 12787 catalog_manager.cc:1347] Generated new cluster ID: a1aaa38e5461465ba3b23d8a75e315c4
I20260806 13:10:42.478435 12787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:42.478842 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:42.473253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:51807 (local address 127.0.0.1:41779)
0806 13:10:42.473404 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:42.473410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:42.473434 (+ 24us) server_negotiation.cc:407] Connection header received
0806 13:10:42.474117 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:42.474127 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:42.474197 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:42.474332 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:42.474553 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:42.475577 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:42.476708 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:42.477362 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:42.477475 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:42.478130 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:42.478133 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:42.478133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:42.478327 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:42.478362 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:42.478377 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:42.478462 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:42.478618 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 13:10:42.478644 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260806 13:10:42.479772 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:42.489203 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:42.482276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:39019 (local address 127.0.0.1:41779)
0806 13:10:42.482309 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:42.482310 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:42.486167 (+ 3857us) server_negotiation.cc:407] Connection header received
0806 13:10:42.486216 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:42.486217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:42.486227 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:42.486277 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:42.486395 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:42.487119 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:42.488047 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:42.488338 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:42.488401 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:42.488980 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:42.488982 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:42.488982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:42.488987 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:42.489001 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:42.489006 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:42.489065 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:42.489122 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 13:10:42.489131 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:10:42.492630 12787 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:42.493193 12787 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:42.504891 12787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 810c271a5c834a0b97a9735d96e12cd2: Generated new TSK 0
I20260806 13:10:42.505036 12787 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:42.514281 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 12671
W20260806 13:10:42.527474 12648 connection.cc:535] client connection to 127.0.0.1:41779 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41779): Connection reset by peer (error 104)
2026-08-06T13:10:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1374 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:42 dist-test-slave-q9h9 krb5kdc[12814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:42 dist-test-slave-q9h9 krb5kdc[12814](info): set up 2 sockets
Aug 06 13:10:42 dist-test-slave-q9h9 krb5kdc[12814](info): commencing operation
krb5kdc: starting...
W20260806 13:10:43.621855 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:43 dist-test-slave-q9h9 krb5kdc[12814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:10:43.728024 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36347
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:43.806107 12831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:43.806286 12831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:43.806315 12831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:43.807423 12831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:43.807466 12831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:43.807485 12831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:43.807502 12831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:43.807518 12831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:43.808970 12831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:43.809206 12831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:43.809405 12831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:43.813807 12845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:43.813813 12843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:43.813815 12839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:43.813841 12831 server_base.cc:1034] running on GCE node
I20260806 13:10:43.814267 12831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:43.814539 12831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:43.815681 12831 hybrid_clock.cc:648] HybridClock initialized: now 1786021843815663 us; error 30 us; skew 500 ppm
Aug 06 13:10:43 dist-test-slave-q9h9 krb5kdc[12814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:43.817860 12831 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:10:43.818298 12831 webserver.cc:460] Webserver started at http://127.0.0.1:40971/ using document root <none> and password file <none>
I20260806 13:10:43.818500 12831 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:43.818548 12831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:43.818674 12831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:43.819605 12831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "ed11f587c5cf4792afcf129dd9ff67e4"
format_stamp: "Formatted at 2026-08-06 13:10:43 on dist-test-slave-q9h9"
server_key: "9194358afa2c52172241bab3c20e98c6"
server_key_iv: "f0ef4efd106fd9173120335bd9a57f01"
server_key_version: "encryptionkey@0"
I20260806 13:10:43.819918 12831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "ed11f587c5cf4792afcf129dd9ff67e4"
format_stamp: "Formatted at 2026-08-06 13:10:43 on dist-test-slave-q9h9"
server_key: "9194358afa2c52172241bab3c20e98c6"
server_key_iv: "f0ef4efd106fd9173120335bd9a57f01"
server_key_version: "encryptionkey@0"
I20260806 13:10:43.821085 12831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 13:10:43.821753 12853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:43.821906 12831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:43.821985 12831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "ed11f587c5cf4792afcf129dd9ff67e4"
format_stamp: "Formatted at 2026-08-06 13:10:43 on dist-test-slave-q9h9"
server_key: "9194358afa2c52172241bab3c20e98c6"
server_key_iv: "f0ef4efd106fd9173120335bd9a57f01"
server_key_version: "encryptionkey@0"
I20260806 13:10:43.822073 12831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:43.825091 12831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:43.826768 12831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:43.826893 12831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:43.830484 12831 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36347
I20260806 13:10:43.830513 12931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36347 every 8 connection(s)
I20260806 13:10:43.830780 12831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:43.831513 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 12831
I20260806 13:10:43.831619 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:43.831663 12933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:43.831799 14209 external_mini_cluster.cc:1428] Setting key bbbe1fa0d006783d086b9099e824b2ec
I20260806 13:10:43.833868 12939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:43.832217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:34475 (local address 127.0.0.1:36347)
0806 13:10:43.832378 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:43.832385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:43.832407 (+ 22us) server_negotiation.cc:407] Connection header received
0806 13:10:43.832821 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:43.832828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:43.832851 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:43.832953 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:43.833532 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:43.833536 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:43.833536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:43.833539 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:43.833573 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:43.833583 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:43.833637 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:43.833716 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 13:10:43.833746 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":61,"threads_started":1}
I20260806 13:10:43.834518 12933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4: Bootstrap starting.
I20260806 13:10:43.834817 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:43.835137 12933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:43.835387 12933 log.cc:824] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:43.835971 12933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4: No bootstrap required, opened a new log
I20260806 13:10:43.837579 12933 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed11f587c5cf4792afcf129dd9ff67e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36347 } }
I20260806 13:10:43.837695 12933 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:43.837726 12933 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed11f587c5cf4792afcf129dd9ff67e4, State: Initialized, Role: FOLLOWER
I20260806 13:10:43.837832 12933 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed11f587c5cf4792afcf129dd9ff67e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36347 } }
I20260806 13:10:43.837899 12933 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:43.837929 12933 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:43.837975 12933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:43.838583 12933 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed11f587c5cf4792afcf129dd9ff67e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36347 } }
I20260806 13:10:43.838694 12933 leader_election.cc:302] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed11f587c5cf4792afcf129dd9ff67e4; no voters:
I20260806 13:10:43.838837 12933 leader_election.cc:288] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:43.838981 12949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:43.839059 12933 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:43.839156 12949 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [term 1 LEADER]: Becoming Leader. State: Replica: ed11f587c5cf4792afcf129dd9ff67e4, State: Running, Role: LEADER
I20260806 13:10:43.839301 12949 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed11f587c5cf4792afcf129dd9ff67e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36347 } }
I20260806 13:10:43.839305 12939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:43.838324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:58237 (local address 127.0.0.1:36347)
0806 13:10:43.838361 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:43.838362 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:43.838428 (+ 66us) server_negotiation.cc:407] Connection header received
0806 13:10:43.838485 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:43.838485 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:43.838489 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:43.838534 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:43.839060 (+ 526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:43.839062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:43.839062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:43.839066 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:43.839078 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:43.839082 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:43.839110 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:43.839239 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 13:10:43.839252 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:10:43.839789 12951 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed11f587c5cf4792afcf129dd9ff67e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed11f587c5cf4792afcf129dd9ff67e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36347 } } }
I20260806 13:10:43.839883 12951 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:43.840080 12952 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed11f587c5cf4792afcf129dd9ff67e4. Latest consensus state: current_term: 1 leader_uuid: "ed11f587c5cf4792afcf129dd9ff67e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed11f587c5cf4792afcf129dd9ff67e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36347 } } }
I20260806 13:10:43.840098 12958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:43.840160 12952 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:43.840548 12958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:43.841955 12958 catalog_manager.cc:1347] Generated new cluster ID: cdad616cd9a1459bb48e3ba14aed266e
I20260806 13:10:43.841997 12958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:43.857892 12958 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:43.858055 12958 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:43.861860 12958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed11f587c5cf4792afcf129dd9ff67e4: Generated new TSK 0
I20260806 13:10:43.862000 12958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:43.867082 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 12831
2026-08-06T13:10:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1354 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:43 dist-test-slave-q9h9 krb5kdc[12980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:43 dist-test-slave-q9h9 krb5kdc[12980](info): set up 2 sockets
Aug 06 13:10:43 dist-test-slave-q9h9 krb5kdc[12980](info): commencing operation
krb5kdc: starting...
W20260806 13:10:44.976212 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:45 dist-test-slave-q9h9 krb5kdc[12980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:10:45.086570 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44915
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:45.164562 13001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:45.164746 13001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:45.164776 13001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:45.165875 13001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:45.165916 13001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:45.165934 13001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:45.165951 13001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:45.165966 13001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:45.167375 13001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44915
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:45.167601 13001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:45.167788 13001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:45.172169 13008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:45.172204 13011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:45.172358 13001 server_base.cc:1034] running on GCE node
W20260806 13:10:45.172173 13009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:45.172648 13001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:45.172904 13001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:45.174049 13001 hybrid_clock.cc:648] HybridClock initialized: now 1786021845174041 us; error 33 us; skew 500 ppm
Aug 06 13:10:45 dist-test-slave-q9h9 krb5kdc[12980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021845, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:45.179901 13001 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:10:45.180296 13001 webserver.cc:460] Webserver started at http://127.0.0.1:46753/ using document root <none> and password file <none>
I20260806 13:10:45.180478 13001 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:45.180526 13001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:45.180647 13001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:45.181571 13001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "4c185596f98d4357bc75094c15759783"
format_stamp: "Formatted at 2026-08-06 13:10:45 on dist-test-slave-q9h9"
server_key: "3261c843608a01fcb1f25bd47912c693"
server_key_iv: "b464f30144f3171b8fb86f130701fcd1"
server_key_version: "encryptionkey@0"
I20260806 13:10:45.181878 13001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "4c185596f98d4357bc75094c15759783"
format_stamp: "Formatted at 2026-08-06 13:10:45 on dist-test-slave-q9h9"
server_key: "3261c843608a01fcb1f25bd47912c693"
server_key_iv: "b464f30144f3171b8fb86f130701fcd1"
server_key_version: "encryptionkey@0"
I20260806 13:10:45.183079 13001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:10:45.183728 13019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:45.183888 13001 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:10:45.183970 13001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "4c185596f98d4357bc75094c15759783"
format_stamp: "Formatted at 2026-08-06 13:10:45 on dist-test-slave-q9h9"
server_key: "3261c843608a01fcb1f25bd47912c693"
server_key_iv: "b464f30144f3171b8fb86f130701fcd1"
server_key_version: "encryptionkey@0"
I20260806 13:10:45.184034 13001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:45.206744 13001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:45.208320 13001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:45.208453 13001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:45.212359 13001 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44915
I20260806 13:10:45.212389 13093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44915 every 8 connection(s)
I20260806 13:10:45.212702 13001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:45.213464 13094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:45.215477 13094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783: Bootstrap starting.
I20260806 13:10:45.216010 13094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:45.216251 13094 log.cc:824] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:45.216883 13094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783: No bootstrap required, opened a new log
I20260806 13:10:45.218356 13094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c185596f98d4357bc75094c15759783" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44915 } }
I20260806 13:10:45.218479 13094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:45.218511 13094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c185596f98d4357bc75094c15759783, State: Initialized, Role: FOLLOWER
I20260806 13:10:45.218641 13094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c185596f98d4357bc75094c15759783" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44915 } }
I20260806 13:10:45.218716 13094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:45.218758 13094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:45.218806 13094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:45.219271 13094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c185596f98d4357bc75094c15759783" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44915 } }
I20260806 13:10:45.219354 13094 leader_election.cc:302] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c185596f98d4357bc75094c15759783; no voters:
I20260806 13:10:45.219496 13094 leader_election.cc:288] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:45.219537 13098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:45.219692 13098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [term 1 LEADER]: Becoming Leader. State: Replica: 4c185596f98d4357bc75094c15759783, State: Running, Role: LEADER
I20260806 13:10:45.219719 13094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:45.219784 13098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c185596f98d4357bc75094c15759783" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44915 } }
I20260806 13:10:45.220142 13102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c185596f98d4357bc75094c15759783. Latest consensus state: current_term: 1 leader_uuid: "4c185596f98d4357bc75094c15759783" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c185596f98d4357bc75094c15759783" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44915 } } }
I20260806 13:10:45.220224 13102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:45.220273 13101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c185596f98d4357bc75094c15759783" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c185596f98d4357bc75094c15759783" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44915 } } }
I20260806 13:10:45.220346 13101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:45.220518 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 13001
I20260806 13:10:45.220629 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:45.220809 14209 external_mini_cluster.cc:1428] Setting key 184be2694aa02bd69bd871fe5338ecb9
I20260806 13:10:45.221028 13113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:45.221501 13113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:45.222908 13113 catalog_manager.cc:1347] Generated new cluster ID: 92e8311978f743b28da02692fdf5aa54
I20260806 13:10:45.222961 13113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 13:10:45 dist-test-slave-q9h9 krb5kdc[12980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 13:10:45.232300 13126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:45.221311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:43817 (local address 127.0.0.1:44915)
0806 13:10:45.222010 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:45.222036 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:45.222069 (+ 33us) server_negotiation.cc:407] Connection header received
0806 13:10:45.222163 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:45.222168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:45.222210 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:45.222457 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:45.223107 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:45.224427 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:45.225446 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:45.226095 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:45.226192 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:45.228766 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:45.228770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:45.228770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:45.228781 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:45.230574 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:45.230997 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:45.230999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:45.231000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:45.231095 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:45.231325 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:45.231325 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:45.231326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:45.231602 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:45.231745 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:45.231867 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 13:10:45.232074 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":70,"threads_started":1}
I20260806 13:10:45.232946 13113 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:45.233352 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:45.233623 13113 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:45.238492 13113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c185596f98d4357bc75094c15759783: Generated new TSK 0
I20260806 13:10:45.238686 13113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:45.244252 13126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:45.238273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:33221 (local address 127.0.0.1:44915)
0806 13:10:45.238372 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:45.238374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:45.238475 (+ 101us) server_negotiation.cc:407] Connection header received
0806 13:10:45.238554 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:45.238554 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:45.238564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:45.238618 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:45.239224 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:45.239698 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:45.240689 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:45.240883 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:45.240933 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:45.242160 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:45.242161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:45.242161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:45.242167 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:45.243292 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:45.243656 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:45.243657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:45.243657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:45.243699 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:45.243885 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:45.243886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:45.243886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:45.243964 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:45.244028 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:45.244159 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 13:10:45.244202 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 13:10:45.246235 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 13001
W20260806 13:10:45.256975 12978 connection.cc:535] client connection to 127.0.0.1:44915 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44915): Connection reset by peer (error 104)
2026-08-06T13:10:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1374 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:45 dist-test-slave-q9h9 krb5kdc[13143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:45 dist-test-slave-q9h9 krb5kdc[13143](info): set up 2 sockets
Aug 06 13:10:45 dist-test-slave-q9h9 krb5kdc[13143](info): commencing operation
krb5kdc: starting...
W20260806 13:10:46.357818 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:46 dist-test-slave-q9h9 krb5kdc[13143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:10:46.465893 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33155
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:46.544714 13159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:46.544905 13159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:46.544938 13159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:46.546082 13159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:46.546125 13159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:46.546147 13159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:46.546166 13159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:46.546186 13159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:46.547634 13159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33155
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:46.547864 13159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:46.548080 13159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:46.552474 13171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:46.552583 13159 server_base.cc:1034] running on GCE node
W20260806 13:10:46.552462 13170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:46.552459 13177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:46.552947 13159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:46.553212 13159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:46.554344 13159 hybrid_clock.cc:648] HybridClock initialized: now 1786021846554336 us; error 29 us; skew 500 ppm
Aug 06 13:10:46 dist-test-slave-q9h9 krb5kdc[13143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021846, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:46.556509 13159 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:10:46.556874 13159 webserver.cc:460] Webserver started at http://127.0.0.1:46379/ using document root <none> and password file <none>
I20260806 13:10:46.557050 13159 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:46.557094 13159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:46.557214 13159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:46.558182 13159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "cdad9b2ad77a4dbd82652e607cdd929c"
format_stamp: "Formatted at 2026-08-06 13:10:46 on dist-test-slave-q9h9"
server_key: "8d2075339206979554d2c6a425fa3402"
server_key_iv: "c4646756b2f1121c5d9963c4dc41d8a2"
server_key_version: "encryptionkey@0"
I20260806 13:10:46.558490 13159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "cdad9b2ad77a4dbd82652e607cdd929c"
format_stamp: "Formatted at 2026-08-06 13:10:46 on dist-test-slave-q9h9"
server_key: "8d2075339206979554d2c6a425fa3402"
server_key_iv: "c4646756b2f1121c5d9963c4dc41d8a2"
server_key_version: "encryptionkey@0"
I20260806 13:10:46.559731 13159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:10:46.560415 13186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:46.560573 13159 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:46.560653 13159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "cdad9b2ad77a4dbd82652e607cdd929c"
format_stamp: "Formatted at 2026-08-06 13:10:46 on dist-test-slave-q9h9"
server_key: "8d2075339206979554d2c6a425fa3402"
server_key_iv: "c4646756b2f1121c5d9963c4dc41d8a2"
server_key_version: "encryptionkey@0"
I20260806 13:10:46.560709 13159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:46.589316 13159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:46.591261 13159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:46.591389 13159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:46.594938 13159 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33155
I20260806 13:10:46.594969 13255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33155 every 8 connection(s)
I20260806 13:10:46.595275 13159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:46.596043 13256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:46.598354 13256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c: Bootstrap starting.
I20260806 13:10:46.598877 13256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:46.599128 13256 log.cc:824] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:46.599763 13256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c: No bootstrap required, opened a new log
I20260806 13:10:46.600010 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 13159
I20260806 13:10:46.600109 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:46.600262 14209 external_mini_cluster.cc:1428] Setting key a70a5f19b82cbdbf7ef8ec8e0fd01e28
I20260806 13:10:46.601696 13256 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdad9b2ad77a4dbd82652e607cdd929c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33155 } }
I20260806 13:10:46.601838 13256 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:46.601861 13256 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdad9b2ad77a4dbd82652e607cdd929c, State: Initialized, Role: FOLLOWER
I20260806 13:10:46.601961 13256 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdad9b2ad77a4dbd82652e607cdd929c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33155 } }
I20260806 13:10:46.602063 13256 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:46.602101 13256 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:46.602146 13256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:46.602661 13256 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdad9b2ad77a4dbd82652e607cdd929c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33155 } }
I20260806 13:10:46.602749 13256 leader_election.cc:302] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdad9b2ad77a4dbd82652e607cdd929c; no voters:
I20260806 13:10:46.602893 13256 leader_election.cc:288] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:46.602916 13262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:46.603027 13262 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [term 1 LEADER]: Becoming Leader. State: Replica: cdad9b2ad77a4dbd82652e607cdd929c, State: Running, Role: LEADER
I20260806 13:10:46.603137 13256 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:46.603163 13262 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdad9b2ad77a4dbd82652e607cdd929c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33155 } }
I20260806 13:10:46.603507 13262 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdad9b2ad77a4dbd82652e607cdd929c. Latest consensus state: current_term: 1 leader_uuid: "cdad9b2ad77a4dbd82652e607cdd929c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdad9b2ad77a4dbd82652e607cdd929c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33155 } } }
I20260806 13:10:46.603740 13262 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:46.603655 13263 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdad9b2ad77a4dbd82652e607cdd929c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdad9b2ad77a4dbd82652e607cdd929c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33155 } } }
I20260806 13:10:46.604173 13263 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:46.604338 13273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:46.604836 13273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:46.606104 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:46.600699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:54605 (local address 127.0.0.1:33155)
0806 13:10:46.600849 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:46.600856 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:46.600879 (+ 23us) server_negotiation.cc:407] Connection header received
0806 13:10:46.601318 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:46.601324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:46.601368 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:46.601471 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:46.601700 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:46.602934 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:46.604007 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:46.604643 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:46.604745 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:46.605416 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:46.605420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:46.605421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:46.605605 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:46.605642 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:46.605657 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:46.605733 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:46.605846 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 13:10:46.605871 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260806 13:10:46.606663 13273 catalog_manager.cc:1347] Generated new cluster ID: 0c0d24bc3c2349ca8aa6081e6cde7766
I20260806 13:10:46.606703 13273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:46.607064 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:46.611785 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:46.608569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:36375 (local address 127.0.0.1:33155)
0806 13:10:46.608604 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:46.608605 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:46.608680 (+ 75us) server_negotiation.cc:407] Connection header received
0806 13:10:46.608738 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:46.608739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:46.608755 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:46.608796 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:46.608921 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:46.609582 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:46.610593 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:46.610892 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:46.610941 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:46.611544 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:46.611546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:46.611546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:46.611550 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:46.611563 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:46.611567 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:46.611610 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:46.611702 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 13:10:46.611712 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:10:46.615200 13273 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:46.615648 13273 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:46.624979 13273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdad9b2ad77a4dbd82652e607cdd929c: Generated new TSK 0
I20260806 13:10:46.625123 13273 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:46.632638 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 13159
W20260806 13:10:46.647017 13141 connection.cc:535] client connection to 127.0.0.1:33155 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33155): Connection reset by peer (error 104)
2026-08-06T13:10:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1389 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:46 dist-test-slave-q9h9 krb5kdc[13297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:46 dist-test-slave-q9h9 krb5kdc[13297](info): set up 2 sockets
Aug 06 13:10:46 dist-test-slave-q9h9 krb5kdc[13297](info): commencing operation
krb5kdc: starting...
W20260806 13:10:47.744134 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:47 dist-test-slave-q9h9 krb5kdc[13297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 13:10:47.849628 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36079
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:47.927070 13321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:47.927242 13321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:47.927263 13321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:47.928342 13321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:47.928382 13321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:47.928395 13321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:47.928408 13321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:47.928432 13321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:47.929840 13321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36079
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:47.930076 13321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:47.930270 13321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:47.934633 13334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:47.934724 13321 server_base.cc:1034] running on GCE node
W20260806 13:10:47.934645 13332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:47.934609 13331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:47.935014 13321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:47.935317 13321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:47.936492 13321 hybrid_clock.cc:648] HybridClock initialized: now 1786021847936478 us; error 31 us; skew 500 ppm
Aug 06 13:10:47 dist-test-slave-q9h9 krb5kdc[13297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:47.938844 13321 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 13:10:47.939234 13321 webserver.cc:460] Webserver started at http://127.0.0.1:33429/ using document root <none> and password file <none>
I20260806 13:10:47.939445 13321 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:47.939493 13321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:47.939610 13321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:47.940529 13321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "879fbc0b40894d7584c0d68a48c3dae2"
format_stamp: "Formatted at 2026-08-06 13:10:47 on dist-test-slave-q9h9"
server_key: "504f5689cc544899902cd66707fa8d6b"
server_key_iv: "f78a2a415b44444571e8e79e326a454d"
server_key_version: "encryptionkey@0"
I20260806 13:10:47.940836 13321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "879fbc0b40894d7584c0d68a48c3dae2"
format_stamp: "Formatted at 2026-08-06 13:10:47 on dist-test-slave-q9h9"
server_key: "504f5689cc544899902cd66707fa8d6b"
server_key_iv: "f78a2a415b44444571e8e79e326a454d"
server_key_version: "encryptionkey@0"
I20260806 13:10:47.941987 13321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:10:47.942633 13344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:47.942778 13321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:47.942857 13321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "879fbc0b40894d7584c0d68a48c3dae2"
format_stamp: "Formatted at 2026-08-06 13:10:47 on dist-test-slave-q9h9"
server_key: "504f5689cc544899902cd66707fa8d6b"
server_key_iv: "f78a2a415b44444571e8e79e326a454d"
server_key_version: "encryptionkey@0"
I20260806 13:10:47.942937 13321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:47.945883 13321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:47.947395 13321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:47.947497 13321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:47.951048 13321 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36079
I20260806 13:10:47.951086 13419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36079 every 8 connection(s)
I20260806 13:10:47.951344 13321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:47.952179 13421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:47.953150 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 13321
I20260806 13:10:47.953239 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:47.953398 14209 external_mini_cluster.cc:1428] Setting key 7a657ca3e67e62b3ba06fc4d2dd0a741
I20260806 13:10:47.954399 13421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2: Bootstrap starting.
I20260806 13:10:47.955001 13421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:47.955283 13421 log.cc:824] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:47.955495 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:47.953818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:44411 (local address 127.0.0.1:36079)
0806 13:10:47.953970 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:47.953977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:47.953999 (+ 22us) server_negotiation.cc:407] Connection header received
0806 13:10:47.954378 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:47.954384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:47.954406 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:47.954526 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:47.955123 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:47.955126 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:47.955127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:47.955137 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:47.955169 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:47.955178 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:47.955248 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:47.955299 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 13:10:47.955338 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260806 13:10:47.955940 13421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2: No bootstrap required, opened a new log
I20260806 13:10:47.956359 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:47.958220 13421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879fbc0b40894d7584c0d68a48c3dae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } }
I20260806 13:10:47.958367 13421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:47.958401 13421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879fbc0b40894d7584c0d68a48c3dae2, State: Initialized, Role: FOLLOWER
I20260806 13:10:47.958562 13421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879fbc0b40894d7584c0d68a48c3dae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } }
I20260806 13:10:47.958662 13421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:47.958706 13421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:47.958756 13421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:47.958840 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:47.957858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:34491 (local address 127.0.0.1:36079)
0806 13:10:47.957906 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:47.957908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:47.957957 (+ 49us) server_negotiation.cc:407] Connection header received
0806 13:10:47.958037 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:47.958038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:47.958043 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:47.958094 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:47.958570 (+ 476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:47.958571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:47.958572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:47.958576 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:47.958589 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:47.958593 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:47.958642 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:47.958780 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 13:10:47.958792 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 13:10:47.959277 13421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879fbc0b40894d7584c0d68a48c3dae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } }
I20260806 13:10:47.959359 13421 leader_election.cc:302] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 879fbc0b40894d7584c0d68a48c3dae2; no voters:
I20260806 13:10:47.959477 13421 leader_election.cc:288] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:47.959511 13433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:47.959649 13433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [term 1 LEADER]: Becoming Leader. State: Replica: 879fbc0b40894d7584c0d68a48c3dae2, State: Running, Role: LEADER
I20260806 13:10:47.959686 13421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:47.959756 13433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879fbc0b40894d7584c0d68a48c3dae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } }
I20260806 13:10:47.960063 13434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "879fbc0b40894d7584c0d68a48c3dae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879fbc0b40894d7584c0d68a48c3dae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } } }
I20260806 13:10:47.960098 13436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 879fbc0b40894d7584c0d68a48c3dae2. Latest consensus state: current_term: 1 leader_uuid: "879fbc0b40894d7584c0d68a48c3dae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "879fbc0b40894d7584c0d68a48c3dae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36079 } } }
I20260806 13:10:47.960253 13436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:47.960179 13434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:47.960446 13445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:47.960927 13445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:47.962299 13445 catalog_manager.cc:1347] Generated new cluster ID: 656fa2fa6b9e47779489098e463fb5de
I20260806 13:10:47.962340 13445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:47.978377 13445 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:47.978510 13445 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:47.981676 13445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 879fbc0b40894d7584c0d68a48c3dae2: Generated new TSK 0
I20260806 13:10:47.981822 13445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:47.987413 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 13321
2026-08-06T13:10:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1352 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:48 dist-test-slave-q9h9 krb5kdc[13460](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:48 dist-test-slave-q9h9 krb5kdc[13460](info): set up 2 sockets
Aug 06 13:10:48 dist-test-slave-q9h9 krb5kdc[13460](info): commencing operation
krb5kdc: starting...
W20260806 13:10:49.086076 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:49 dist-test-slave-q9h9 krb5kdc[13460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:10:49.192476 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:41285
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:41285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:49.271071 13484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:49.271255 13484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:49.271284 13484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:49.272377 13484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:49.272418 13484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:49.272436 13484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:49.272452 13484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:49.272467 13484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:49.273873 13484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:41285
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:41285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:49.274125 13484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:49.274322 13484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:49.278784 13492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:49.278805 13499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:49.278849 13484 server_base.cc:1034] running on GCE node
W20260806 13:10:49.278928 13502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:49.279134 13484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:49.279383 13484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:49.280519 13484 hybrid_clock.cc:648] HybridClock initialized: now 1786021849280506 us; error 31 us; skew 500 ppm
Aug 06 13:10:49 dist-test-slave-q9h9 krb5kdc[13460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021849, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:49.282779 13484 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:10:49.283181 13484 webserver.cc:460] Webserver started at http://127.13.224.126:42587/ using document root <none> and password file <none>
I20260806 13:10:49.283365 13484 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:49.283406 13484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:49.283525 13484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:49.284452 13484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "700a218e1cd54bb385c071d77aea7e24"
format_stamp: "Formatted at 2026-08-06 13:10:49 on dist-test-slave-q9h9"
server_key: "6e8fcfb6ba530fecd4756250780f7707"
server_key_iv: "322e616f165af9a6347f4f5c6bf15cd8"
server_key_version: "encryptionkey@0"
I20260806 13:10:49.284770 13484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "700a218e1cd54bb385c071d77aea7e24"
format_stamp: "Formatted at 2026-08-06 13:10:49 on dist-test-slave-q9h9"
server_key: "6e8fcfb6ba530fecd4756250780f7707"
server_key_iv: "322e616f165af9a6347f4f5c6bf15cd8"
server_key_version: "encryptionkey@0"
I20260806 13:10:49.285928 13484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:10:49.286636 13512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:49.286773 13484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:49.286849 13484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "700a218e1cd54bb385c071d77aea7e24"
format_stamp: "Formatted at 2026-08-06 13:10:49 on dist-test-slave-q9h9"
server_key: "6e8fcfb6ba530fecd4756250780f7707"
server_key_iv: "322e616f165af9a6347f4f5c6bf15cd8"
server_key_version: "encryptionkey@0"
I20260806 13:10:49.286916 13484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:49.310740 13484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:49.312302 13484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:49.312425 13484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:49.316155 13484 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:41285
I20260806 13:10:49.316190 13584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:41285 every 8 connection(s)
I20260806 13:10:49.316510 13484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:49.316632 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 13484
I20260806 13:10:49.316744 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:49.316949 14209 external_mini_cluster.cc:1428] Setting key 44a5e59c907925c6fe5f487a52255d2d
I20260806 13:10:49.317481 13589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:49.320221 13589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24: Bootstrap starting.
I20260806 13:10:49.320897 13589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:49.321151 13589 log.cc:824] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:49.321843 13589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24: No bootstrap required, opened a new log
Aug 06 13:10:49 dist-test-slave-q9h9 krb5kdc[13460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:49.323992 13589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "700a218e1cd54bb385c071d77aea7e24" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41285 } }
I20260806 13:10:49.324119 13589 raft_consensus.cc:374] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:49.324154 13589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 700a218e1cd54bb385c071d77aea7e24, State: Initialized, Role: FOLLOWER
I20260806 13:10:49.324281 13589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "700a218e1cd54bb385c071d77aea7e24" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41285 } }
I20260806 13:10:49.324345 13589 raft_consensus.cc:388] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:49.324368 13589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:49.324412 13589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:49.324891 13589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "700a218e1cd54bb385c071d77aea7e24" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41285 } }
I20260806 13:10:49.324975 13589 leader_election.cc:302] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 700a218e1cd54bb385c071d77aea7e24; no voters:
I20260806 13:10:49.325126 13589 leader_election.cc:288] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:49.325174 13596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:49.325323 13596 raft_consensus.cc:686] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [term 1 LEADER]: Becoming Leader. State: Replica: 700a218e1cd54bb385c071d77aea7e24, State: Running, Role: LEADER
I20260806 13:10:49.325362 13589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:49.325515 13596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "700a218e1cd54bb385c071d77aea7e24" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41285 } }
I20260806 13:10:49.326520 13597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 700a218e1cd54bb385c071d77aea7e24. Latest consensus state: current_term: 1 leader_uuid: "700a218e1cd54bb385c071d77aea7e24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "700a218e1cd54bb385c071d77aea7e24" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41285 } } }
I20260806 13:10:49.326598 13597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [sys.catalog]: This master's current role is: LEADER
W20260806 13:10:49.327134 13618 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:10:49.327176 13618 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:10:49.327273 13619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:49.327414 13619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:49.327696 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:49.317405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53177 (local address 127.13.224.126:41285)
0806 13:10:49.317565 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:49.317573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:49.317596 (+ 23us) server_negotiation.cc:407] Connection header received
0806 13:10:49.318005 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:49.318011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:49.318085 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:49.318191 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:49.318922 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:49.319762 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:49.320769 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:49.321170 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:49.321258 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:49.323802 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:49.323806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:49.323806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:49.323816 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:49.325562 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:49.326003 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:49.326006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:49.326007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:49.326147 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:49.326464 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:49.326465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:49.326466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:49.326779 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:49.326924 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:49.327411 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 13:10:49.327511 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260806 13:10:49.328279 13598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "700a218e1cd54bb385c071d77aea7e24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "700a218e1cd54bb385c071d77aea7e24" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 41285 } } }
I20260806 13:10:49.328411 13598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:49.328644 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:49.334932 13619 catalog_manager.cc:1347] Generated new cluster ID: f6a0a907a3914e8e91959e41b476ab8e
I20260806 13:10:49.334972 13619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:49.336119 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:49.329883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55911 (local address 127.13.224.126:41285)
0806 13:10:49.329918 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:49.329920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:49.329969 (+ 49us) server_negotiation.cc:407] Connection header received
0806 13:10:49.330059 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:49.330060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:49.330068 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:49.330112 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:49.330718 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:49.331168 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:49.332127 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:49.332316 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:49.332358 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:49.333568 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:49.333569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:49.333570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:49.333575 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:49.334826 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:49.335252 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:49.335253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:49.335253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:49.335309 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:49.335547 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:49.335548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:49.335548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:49.335664 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:49.335734 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:49.335968 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 13:10:49.336030 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:10:49.356917 13619 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:49.357393 13619 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:49.362772 13619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 700a218e1cd54bb385c071d77aea7e24: Generated new TSK 0
I20260806 13:10:49.362921 13619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:49.366375 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 13484
W20260806 13:10:49.378983 13455 connection.cc:535] client connection to 127.13.224.126:41285 recv error: Network error: failed to read from TLS socket (remote: 127.13.224.126:41285): Connection reset by peer (error 104)
2026-08-06T13:10:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:49 dist-test-slave-q9h9 krb5kdc[13631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:49 dist-test-slave-q9h9 krb5kdc[13631](info): set up 2 sockets
Aug 06 13:10:49 dist-test-slave-q9h9 krb5kdc[13631](info): commencing operation
krb5kdc: starting...
W20260806 13:10:50.465723 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:50 dist-test-slave-q9h9 krb5kdc[13631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:10:50.570385 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:42767
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:43111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:42767
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:50.648447 13655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:50.648624 13655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:50.648653 13655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:50.649747 13655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:50.649786 13655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:50.649806 13655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:50.649820 13655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:50.649837 13655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:50.651304 13655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:43111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:42767
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:42767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:50.651523 13655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:50.651721 13655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:50.656107 13662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:50.656126 13664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:50.656215 13655 server_base.cc:1034] running on GCE node
W20260806 13:10:50.656111 13666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:50.656549 13655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:50.656800 13655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:50.657930 13655 hybrid_clock.cc:648] HybridClock initialized: now 1786021850657918 us; error 29 us; skew 500 ppm
Aug 06 13:10:50 dist-test-slave-q9h9 krb5kdc[13631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:50.664067 13655 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:10:50.664474 13655 webserver.cc:460] Webserver started at http://127.13.224.126:42919/ using document root <none> and password file <none>
I20260806 13:10:50.664659 13655 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:50.664700 13655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:50.664786 13655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:50.665676 13655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "7ebbb62f8a0245f3becef4ea5c7fcb95"
format_stamp: "Formatted at 2026-08-06 13:10:50 on dist-test-slave-q9h9"
server_key: "efe160b1d132144239580cf058a0ff42"
server_key_iv: "f50e20e7a09f56cf6685159253a5e226"
server_key_version: "encryptionkey@0"
I20260806 13:10:50.665956 13655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "7ebbb62f8a0245f3becef4ea5c7fcb95"
format_stamp: "Formatted at 2026-08-06 13:10:50 on dist-test-slave-q9h9"
server_key: "efe160b1d132144239580cf058a0ff42"
server_key_iv: "f50e20e7a09f56cf6685159253a5e226"
server_key_version: "encryptionkey@0"
I20260806 13:10:50.667106 13655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:10:50.667723 13675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:50.667869 13655 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:10:50.667944 13655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "7ebbb62f8a0245f3becef4ea5c7fcb95"
format_stamp: "Formatted at 2026-08-06 13:10:50 on dist-test-slave-q9h9"
server_key: "efe160b1d132144239580cf058a0ff42"
server_key_iv: "f50e20e7a09f56cf6685159253a5e226"
server_key_version: "encryptionkey@0"
I20260806 13:10:50.668006 13655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:50.676934 13655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:50.678442 13655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:50.678551 13655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:50.682168 13655 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:42767
I20260806 13:10:50.682195 13761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:42767 every 8 connection(s)
I20260806 13:10:50.682487 13655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:50.683233 13762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:50.684110 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 13655
I20260806 13:10:50.684199 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:50.684365 14209 external_mini_cluster.cc:1428] Setting key c5cb4a9bfb183e68137226da728ad568
I20260806 13:10:50.685564 13762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95: Bootstrap starting.
I20260806 13:10:50.686182 13762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:50.686436 13762 log.cc:824] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:50.687033 13762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95: No bootstrap required, opened a new log
I20260806 13:10:50.688782 13762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ebbb62f8a0245f3becef4ea5c7fcb95" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42767 } }
I20260806 13:10:50.688906 13762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:50.688939 13762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ebbb62f8a0245f3becef4ea5c7fcb95, State: Initialized, Role: FOLLOWER
I20260806 13:10:50.689060 13762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ebbb62f8a0245f3becef4ea5c7fcb95" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42767 } }
I20260806 13:10:50.689142 13762 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:50.689201 13762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:50.689262 13762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:50.689335 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:50.684787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52035 (local address 127.13.224.126:42767)
0806 13:10:50.684953 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:50.684961 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:50.684989 (+ 28us) server_negotiation.cc:407] Connection header received
0806 13:10:50.685383 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:50.685391 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:50.685450 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:50.685562 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:50.685780 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:50.686753 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:50.687534 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:50.687925 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:50.688017 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:50.688649 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:50.688653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:50.688653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:50.688807 (+ 154us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:50.688833 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:50.688843 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:50.688908 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:50.689072 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 13:10:50.689095 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":58,"threads_started":1}
I20260806 13:10:50.689903 13762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ebbb62f8a0245f3becef4ea5c7fcb95" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42767 } }
I20260806 13:10:50.690009 13762 leader_election.cc:302] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ebbb62f8a0245f3becef4ea5c7fcb95; no voters:
I20260806 13:10:50.690186 13762 leader_election.cc:288] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:50.690212 13770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:50.690217 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:50.690327 13770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [term 1 LEADER]: Becoming Leader. State: Replica: 7ebbb62f8a0245f3becef4ea5c7fcb95, State: Running, Role: LEADER
I20260806 13:10:50.690419 13770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ebbb62f8a0245f3becef4ea5c7fcb95" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42767 } }
I20260806 13:10:50.690418 13762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:50.690754 13770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ebbb62f8a0245f3becef4ea5c7fcb95. Latest consensus state: current_term: 1 leader_uuid: "7ebbb62f8a0245f3becef4ea5c7fcb95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ebbb62f8a0245f3becef4ea5c7fcb95" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42767 } } }
I20260806 13:10:50.690768 13771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ebbb62f8a0245f3becef4ea5c7fcb95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ebbb62f8a0245f3becef4ea5c7fcb95" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 42767 } } }
I20260806 13:10:50.690866 13770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:50.690950 13771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:50.691184 13781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:50.691844 13781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:50.693868 13781 catalog_manager.cc:1347] Generated new cluster ID: 4e74d06f12874c3880f90cdc0b6079bd
I20260806 13:10:50.693955 13781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:50.695024 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:50.691754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54837 (local address 127.13.224.126:42767)
0806 13:10:50.691789 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:50.691791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:50.692097 (+ 306us) server_negotiation.cc:407] Connection header received
0806 13:10:50.692234 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:50.692235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:50.692245 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:50.692303 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:50.692457 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:50.692969 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:50.693821 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:50.694094 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:50.694153 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:50.694789 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:50.694790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:50.694791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:50.694795 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:50.694819 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:50.694824 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:50.694887 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:50.694946 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 13:10:50.694956 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:10:50.697958 13781 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:50.698643 13781 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:50.706753 13781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ebbb62f8a0245f3becef4ea5c7fcb95: Generated new TSK 0
I20260806 13:10:50.706902 13781 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:50.708842 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 13655
W20260806 13:10:50.718773 13628 connection.cc:535] client connection to 127.13.224.126:42767 recv error: Network error: failed to read from TLS socket (remote: 127.13.224.126:42767): Connection reset by peer (error 104)
2026-08-06T13:10:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:50 dist-test-slave-q9h9 krb5kdc[13808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:50 dist-test-slave-q9h9 krb5kdc[13808](info): set up 2 sockets
Aug 06 13:10:50 dist-test-slave-q9h9 krb5kdc[13808](info): commencing operation
krb5kdc: starting...
W20260806 13:10:51.813658 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:51 dist-test-slave-q9h9 krb5kdc[13808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:10:51.920104 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:36729
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:36729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:51.997527 13823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:51.997704 13823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:51.997725 13823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:51.998782 13823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:51.998816 13823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:51.998829 13823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:51.998840 13823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:51.998853 13823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:52.000217 13823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:36729
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:36729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:52.000380 13823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:52.000545 13823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:52.004894 13838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:52.004908 13835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:52.004940 13823 server_base.cc:1034] running on GCE node
W20260806 13:10:52.004914 13833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:52.005360 13823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:52.005615 13823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:52.006754 13823 hybrid_clock.cc:648] HybridClock initialized: now 1786021852006738 us; error 29 us; skew 500 ppm
Aug 06 13:10:52 dist-test-slave-q9h9 krb5kdc[13808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021852, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:52.008934 13823 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:10:52.009327 13823 webserver.cc:460] Webserver started at http://127.13.224.126:36169/ using document root <none> and password file <none>
I20260806 13:10:52.009521 13823 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:52.009567 13823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:52.009686 13823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:52.010648 13823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "9ccabd75d92f4050b7a43302be706f8f"
format_stamp: "Formatted at 2026-08-06 13:10:52 on dist-test-slave-q9h9"
server_key: "6b34c29777edd82ba48592fa83fb37c2"
server_key_iv: "1e3fe2406dfc6877d2428c1387a56a58"
server_key_version: "encryptionkey@0"
I20260806 13:10:52.010960 13823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "9ccabd75d92f4050b7a43302be706f8f"
format_stamp: "Formatted at 2026-08-06 13:10:52 on dist-test-slave-q9h9"
server_key: "6b34c29777edd82ba48592fa83fb37c2"
server_key_iv: "1e3fe2406dfc6877d2428c1387a56a58"
server_key_version: "encryptionkey@0"
I20260806 13:10:52.012094 13823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:52.012699 13846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:52.012842 13823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:52.012920 13823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "9ccabd75d92f4050b7a43302be706f8f"
format_stamp: "Formatted at 2026-08-06 13:10:52 on dist-test-slave-q9h9"
server_key: "6b34c29777edd82ba48592fa83fb37c2"
server_key_iv: "1e3fe2406dfc6877d2428c1387a56a58"
server_key_version: "encryptionkey@0"
I20260806 13:10:52.012979 13823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:52.016027 13823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:52.017534 13823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:52.017652 13823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:52.021193 13823 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:36729
I20260806 13:10:52.021224 13923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:36729 every 8 connection(s)
I20260806 13:10:52.021489 13823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:52.022378 13926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:52.023689 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 13823
I20260806 13:10:52.023810 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:52.023993 14209 external_mini_cluster.cc:1428] Setting key 411ee8bd5dc7f2018eafb8d0a9d11de8
I20260806 13:10:52.024601 13926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f: Bootstrap starting.
I20260806 13:10:52.025141 13926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:52.025455 13926 log.cc:824] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:52.026170 13931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:52.024439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59521 (local address 127.13.224.126:36729)
0806 13:10:52.024586 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:52.024593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:52.024615 (+ 22us) server_negotiation.cc:407] Connection header received
0806 13:10:52.025011 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:52.025017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:52.025041 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:52.025145 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:52.025745 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:52.025748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:52.025748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:52.025753 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:52.025795 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:52.025809 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:52.025882 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:52.025940 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 13:10:52.025973 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260806 13:10:52.026190 13926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f: No bootstrap required, opened a new log
I20260806 13:10:52.027032 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:52.028210 13926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccabd75d92f4050b7a43302be706f8f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36729 } }
I20260806 13:10:52.028373 13926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:52.028417 13926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ccabd75d92f4050b7a43302be706f8f, State: Initialized, Role: FOLLOWER
I20260806 13:10:52.028529 13926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccabd75d92f4050b7a43302be706f8f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36729 } }
I20260806 13:10:52.028609 13926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:52.028635 13926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:52.028677 13926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:52.029150 13926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccabd75d92f4050b7a43302be706f8f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36729 } }
I20260806 13:10:52.029217 13931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:52.028295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35689 (local address 127.13.224.126:36729)
0806 13:10:52.028338 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:52.028340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:52.028436 (+ 96us) server_negotiation.cc:407] Connection header received
0806 13:10:52.028518 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:52.028519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:52.028523 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:52.028557 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:52.029045 (+ 488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:52.029046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:52.029046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:52.029050 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:52.029061 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:52.029065 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:52.029109 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:52.029152 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 13:10:52.029164 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 13:10:52.029285 13926 leader_election.cc:302] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ccabd75d92f4050b7a43302be706f8f; no voters:
I20260806 13:10:52.029428 13926 leader_election.cc:288] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:52.029460 13943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:52.029594 13943 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [term 1 LEADER]: Becoming Leader. State: Replica: 9ccabd75d92f4050b7a43302be706f8f, State: Running, Role: LEADER
I20260806 13:10:52.029661 13926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:52.029680 13943 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccabd75d92f4050b7a43302be706f8f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36729 } }
I20260806 13:10:52.029989 13943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ccabd75d92f4050b7a43302be706f8f. Latest consensus state: current_term: 1 leader_uuid: "9ccabd75d92f4050b7a43302be706f8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccabd75d92f4050b7a43302be706f8f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36729 } } }
I20260806 13:10:52.029996 13947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ccabd75d92f4050b7a43302be706f8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccabd75d92f4050b7a43302be706f8f" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36729 } } }
I20260806 13:10:52.030141 13943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:52.030236 13947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:52.030668 13957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:52.031165 13957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:52.032460 13957 catalog_manager.cc:1347] Generated new cluster ID: 99b99015fef1444b9ae4190ce2797f7d
I20260806 13:10:52.032510 13957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:52.039624 13957 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:52.039765 13957 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:52.050832 13957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ccabd75d92f4050b7a43302be706f8f: Generated new TSK 0
I20260806 13:10:52.050983 13957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:52.058873 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 13823
2026-08-06T13:10:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1353 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:52 dist-test-slave-q9h9 krb5kdc[13977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:52 dist-test-slave-q9h9 krb5kdc[13977](info): set up 2 sockets
Aug 06 13:10:52 dist-test-slave-q9h9 krb5kdc[13977](info): commencing operation
krb5kdc: starting...
W20260806 13:10:53.163280 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:53 dist-test-slave-q9h9 krb5kdc[13977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:10:53.266429 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:34623
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:33545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:34623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:53.344290 13993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:53.344467 13993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:53.344498 13993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:53.345577 13993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:53.345616 13993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:53.345635 13993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:53.345650 13993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:53.345667 13993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:53.347151 13993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:33545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:34623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:34623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:53.347368 13993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:53.347569 13993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:53.351984 14000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:53.352030 13993 server_base.cc:1034] running on GCE node
W20260806 13:10:53.352057 13999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:53.351997 14002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:53.352322 13993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:53.352559 13993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:53.353689 13993 hybrid_clock.cc:648] HybridClock initialized: now 1786021853353674 us; error 33 us; skew 500 ppm
Aug 06 13:10:53 dist-test-slave-q9h9 krb5kdc[13977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021853, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:53.355876 13993 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:10:53.356267 13993 webserver.cc:460] Webserver started at http://127.13.224.126:33611/ using document root <none> and password file <none>
I20260806 13:10:53.356468 13993 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:53.356509 13993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:53.356624 13993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:53.357546 13993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "752923803aba43afafc3edae82818d06"
format_stamp: "Formatted at 2026-08-06 13:10:53 on dist-test-slave-q9h9"
server_key: "7f93ebf568c081854641d2fe3f7f2553"
server_key_iv: "117a31b2052896e985e6ff210e5dea4f"
server_key_version: "encryptionkey@0"
I20260806 13:10:53.357853 13993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "752923803aba43afafc3edae82818d06"
format_stamp: "Formatted at 2026-08-06 13:10:53 on dist-test-slave-q9h9"
server_key: "7f93ebf568c081854641d2fe3f7f2553"
server_key_iv: "117a31b2052896e985e6ff210e5dea4f"
server_key_version: "encryptionkey@0"
I20260806 13:10:53.359038 13993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 13:10:53.359638 14011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:53.359807 13993 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:53.359877 13993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "752923803aba43afafc3edae82818d06"
format_stamp: "Formatted at 2026-08-06 13:10:53 on dist-test-slave-q9h9"
server_key: "7f93ebf568c081854641d2fe3f7f2553"
server_key_iv: "117a31b2052896e985e6ff210e5dea4f"
server_key_version: "encryptionkey@0"
I20260806 13:10:53.359926 13993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:53.372099 13993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:53.373881 13993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:53.373979 13993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:53.377473 13993 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:34623
I20260806 13:10:53.377506 14088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:34623 every 8 connection(s)
I20260806 13:10:53.377784 13993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:53.378535 14089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:53.380059 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 13993
I20260806 13:10:53.380151 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:53.380313 14209 external_mini_cluster.cc:1428] Setting key 55b9c1df42eaabaf6c6bf8d415550f79
I20260806 13:10:53.380584 14089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06: Bootstrap starting.
I20260806 13:10:53.381248 14089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:53.381564 14089 log.cc:824] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:53.382300 14089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06: No bootstrap required, opened a new log
I20260806 13:10:53.383939 14089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752923803aba43afafc3edae82818d06" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34623 } }
I20260806 13:10:53.384054 14089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:53.384075 14089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 752923803aba43afafc3edae82818d06, State: Initialized, Role: FOLLOWER
I20260806 13:10:53.384186 14089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752923803aba43afafc3edae82818d06" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34623 } }
I20260806 13:10:53.384243 14089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:53.384267 14089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:53.384310 14089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:53.384785 14089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752923803aba43afafc3edae82818d06" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34623 } }
I20260806 13:10:53.384881 14089 leader_election.cc:302] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 752923803aba43afafc3edae82818d06; no voters:
I20260806 13:10:53.385026 14089 leader_election.cc:288] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:53.385121 14096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:53.385250 14089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:53.385329 14096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [term 1 LEADER]: Becoming Leader. State: Replica: 752923803aba43afafc3edae82818d06, State: Running, Role: LEADER
I20260806 13:10:53.385581 14096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752923803aba43afafc3edae82818d06" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34623 } }
I20260806 13:10:53.386122 14096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 752923803aba43afafc3edae82818d06. Latest consensus state: current_term: 1 leader_uuid: "752923803aba43afafc3edae82818d06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752923803aba43afafc3edae82818d06" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34623 } } }
I20260806 13:10:53.386281 14096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:53.386360 14097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "752923803aba43afafc3edae82818d06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "752923803aba43afafc3edae82818d06" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 34623 } } }
I20260806 13:10:53.386479 14097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06 [sys.catalog]: This master's current role is: LEADER
Aug 06 13:10:53 dist-test-slave-q9h9 krb5kdc[13977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:53.387048 14108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:53.387806 14108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:53.389286 14108 catalog_manager.cc:1347] Generated new cluster ID: 5ff3782329de46818d0a2e04beffa210
I20260806 13:10:53.389348 14108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:53.390406 14093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:53.380737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47325 (local address 127.13.224.126:34623)
0806 13:10:53.381009 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:53.381017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:53.381046 (+ 29us) server_negotiation.cc:407] Connection header received
0806 13:10:53.381568 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:53.381577 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:53.381633 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:53.381773 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:53.382461 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:53.383306 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:53.384027 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:53.384555 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:53.384645 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:53.387274 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:53.387278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:53.387278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:53.387288 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:53.388962 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:53.389290 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:53.389293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:53.389293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:53.389413 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:53.389607 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:53.389608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:53.389608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:53.389823 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:53.389959 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:53.390089 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 13:10:53.390212 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":70,"threads_started":1}
I20260806 13:10:53.391189 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:53.402132 14108 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:53.402592 14108 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:53.403600 14093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:53.394308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56697 (local address 127.13.224.126:34623)
0806 13:10:53.394345 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:53.394347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:53.398224 (+ 3877us) server_negotiation.cc:407] Connection header received
0806 13:10:53.398270 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:53.398271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:53.398280 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:53.398322 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:53.398775 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:53.399219 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:53.399859 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:53.400062 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:53.400115 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:53.401084 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:53.401085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:53.401085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:53.401092 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:53.402521 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:53.402896 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:53.402897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:53.402897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:53.402957 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:53.403123 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:53.403123 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:53.403124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:53.403239 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:53.403326 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:53.403436 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 13:10:53.403517 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:10:53.407121 14108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 752923803aba43afafc3edae82818d06: Generated new TSK 0
I20260806 13:10:53.407258 14108 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:53.408928 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 13993
2026-08-06T13:10:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:53 dist-test-slave-q9h9 krb5kdc[14133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:53 dist-test-slave-q9h9 krb5kdc[14133](info): set up 2 sockets
Aug 06 13:10:53 dist-test-slave-q9h9 krb5kdc[14133](info): commencing operation
krb5kdc: starting...
W20260806 13:10:54.522729 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:54 dist-test-slave-q9h9 krb5kdc[14133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:10:54.626319 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:43595
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:41009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:43595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:54.703732 14158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:54.703919 14158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:54.703948 14158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:54.705030 14158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:54.705070 14158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:54.705087 14158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:54.705103 14158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:54.705119 14158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:54.706548 14158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:41009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:43595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:43595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:54.706785 14158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:54.706975 14158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:54.711366 14170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:54.711376 14165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:54.711519 14168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:54.711474 14158 server_base.cc:1034] running on GCE node
I20260806 13:10:54.711781 14158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:54.712016 14158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:54.713141 14158 hybrid_clock.cc:648] HybridClock initialized: now 1786021854713133 us; error 29 us; skew 500 ppm
Aug 06 13:10:54 dist-test-slave-q9h9 krb5kdc[14133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021854, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:54.715353 14158 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:10:54.715713 14158 webserver.cc:460] Webserver started at http://127.13.224.126:46261/ using document root <none> and password file <none>
I20260806 13:10:54.715893 14158 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:54.715930 14158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:54.716015 14158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:54.716895 14158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "7665387286144613b5b6bf36d15846f2"
format_stamp: "Formatted at 2026-08-06 13:10:54 on dist-test-slave-q9h9"
server_key: "39e036613ffcc7f2f2fb68d81393d07e"
server_key_iv: "589c098ec53e3c3b39953fa194f0707b"
server_key_version: "encryptionkey@0"
I20260806 13:10:54.717164 14158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "7665387286144613b5b6bf36d15846f2"
format_stamp: "Formatted at 2026-08-06 13:10:54 on dist-test-slave-q9h9"
server_key: "39e036613ffcc7f2f2fb68d81393d07e"
server_key_iv: "589c098ec53e3c3b39953fa194f0707b"
server_key_version: "encryptionkey@0"
I20260806 13:10:54.718287 14158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 13:10:54.718892 14181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:54.719030 14158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:10:54.719111 14158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "7665387286144613b5b6bf36d15846f2"
format_stamp: "Formatted at 2026-08-06 13:10:54 on dist-test-slave-q9h9"
server_key: "39e036613ffcc7f2f2fb68d81393d07e"
server_key_iv: "589c098ec53e3c3b39953fa194f0707b"
server_key_version: "encryptionkey@0"
I20260806 13:10:54.719168 14158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:54.738361 14158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:54.739833 14158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:54.739953 14158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:54.743495 14158 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:43595
I20260806 13:10:54.743533 14264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:43595 every 8 connection(s)
I20260806 13:10:54.743808 14158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:54.744537 14265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:54.746503 14265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2: Bootstrap starting.
I20260806 13:10:54.747015 14265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:54.747253 14265 log.cc:824] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:54.747839 14265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2: No bootstrap required, opened a new log
I20260806 13:10:54.749275 14265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7665387286144613b5b6bf36d15846f2" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43595 } }
I20260806 13:10:54.749395 14265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:54.749428 14265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7665387286144613b5b6bf36d15846f2, State: Initialized, Role: FOLLOWER
I20260806 13:10:54.749559 14265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7665387286144613b5b6bf36d15846f2" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43595 } }
I20260806 13:10:54.749630 14265 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:54.749662 14265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:54.749708 14265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:54.750094 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 14158
I20260806 13:10:54.750200 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:54.750386 14209 external_mini_cluster.cc:1428] Setting key 13ca1c4b15d6edd8d8d142f239b9fa54
I20260806 13:10:54.750648 14265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7665387286144613b5b6bf36d15846f2" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43595 } }
I20260806 13:10:54.750748 14265 leader_election.cc:302] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7665387286144613b5b6bf36d15846f2; no voters:
I20260806 13:10:54.750919 14265 leader_election.cc:288] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:54.751061 14275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:54.751127 14265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:54.751186 14275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [term 1 LEADER]: Becoming Leader. State: Replica: 7665387286144613b5b6bf36d15846f2, State: Running, Role: LEADER
I20260806 13:10:54.751294 14275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7665387286144613b5b6bf36d15846f2" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43595 } }
I20260806 13:10:54.751605 14275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7665387286144613b5b6bf36d15846f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7665387286144613b5b6bf36d15846f2" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43595 } } }
I20260806 13:10:54.751683 14275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:54.751820 14275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7665387286144613b5b6bf36d15846f2. Latest consensus state: current_term: 1 leader_uuid: "7665387286144613b5b6bf36d15846f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7665387286144613b5b6bf36d15846f2" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 43595 } } }
I20260806 13:10:54.751874 14275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:54.751961 14285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:54.752491 14285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:54.753930 14285 catalog_manager.cc:1347] Generated new cluster ID: fde88c4007e040249df7d0729aeebec5
I20260806 13:10:54.753976 14285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:54.756100 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:54.750805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38827 (local address 127.13.224.126:43595)
0806 13:10:54.750979 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:54.750988 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:54.751018 (+ 30us) server_negotiation.cc:407] Connection header received
0806 13:10:54.751128 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:54.751134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:54.751192 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:54.751337 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:54.751664 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:54.753137 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:54.754201 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:54.754773 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:54.754881 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:54.755488 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:54.755492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:54.755492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:54.755672 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:54.755705 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:54.755718 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:54.755775 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:54.755864 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 13:10:54.755921 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260806 13:10:54.757071 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:54.761512 14285 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:54.761683 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:54.758405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33069 (local address 127.13.224.126:43595)
0806 13:10:54.758436 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:54.758438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:54.758533 (+ 95us) server_negotiation.cc:407] Connection header received
0806 13:10:54.758605 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:54.758606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:54.758614 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:54.758636 (+ 22us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:54.758790 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:54.759471 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:54.760421 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:54.760714 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:54.760750 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:54.761419 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:54.761420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:54.761420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:54.761425 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:54.761436 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:54.761440 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:54.761482 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:54.761602 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 13:10:54.761622 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260806 13:10:54.762158 14285 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:54.774468 14285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7665387286144613b5b6bf36d15846f2: Generated new TSK 0
I20260806 13:10:54.774621 14285 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:54.776928 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 14158
2026-08-06T13:10:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1371 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:54 dist-test-slave-q9h9 krb5kdc[14311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:54 dist-test-slave-q9h9 krb5kdc[14311](info): set up 2 sockets
Aug 06 13:10:54 dist-test-slave-q9h9 krb5kdc[14311](info): commencing operation
krb5kdc: starting...
W20260806 13:10:55.896422 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:55 dist-test-slave-q9h9 krb5kdc[14311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:10:56.002364 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35221
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:35221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:56.082485 14331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:56.082676 14331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:56.082710 14331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:56.083820 14331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:56.083868 14331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:56.083890 14331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:56.083909 14331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:56.083928 14331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:56.085384 14331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:35221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:35221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:56.085624 14331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:56.085822 14331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:56.090258 14339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:56.090284 14342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:56.090284 14338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:56.090461 14331 server_base.cc:1034] running on GCE node
I20260806 13:10:56.090768 14331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:56.091040 14331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:56.092176 14331 hybrid_clock.cc:648] HybridClock initialized: now 1786021856092165 us; error 30 us; skew 500 ppm
Aug 06 13:10:56 dist-test-slave-q9h9 krb5kdc[14311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021856, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:56.094502 14331 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:10:56.094913 14331 webserver.cc:460] Webserver started at http://127.13.224.126:44859/ using document root <none> and password file <none>
I20260806 13:10:56.095098 14331 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:56.095144 14331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:56.095263 14331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:56.096204 14331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "057763197c4c47b8854d33e415049239"
format_stamp: "Formatted at 2026-08-06 13:10:56 on dist-test-slave-q9h9"
server_key: "44a059d3f5640e684d87e4bdaf178352"
server_key_iv: "1c6b1108a539707799cfa07159c0d21e"
server_key_version: "encryptionkey@0"
I20260806 13:10:56.096522 14331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "057763197c4c47b8854d33e415049239"
format_stamp: "Formatted at 2026-08-06 13:10:56 on dist-test-slave-q9h9"
server_key: "44a059d3f5640e684d87e4bdaf178352"
server_key_iv: "1c6b1108a539707799cfa07159c0d21e"
server_key_version: "encryptionkey@0"
I20260806 13:10:56.097693 14331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:56.098351 14349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:56.098505 14331 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:56.098582 14331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "057763197c4c47b8854d33e415049239"
format_stamp: "Formatted at 2026-08-06 13:10:56 on dist-test-slave-q9h9"
server_key: "44a059d3f5640e684d87e4bdaf178352"
server_key_iv: "1c6b1108a539707799cfa07159c0d21e"
server_key_version: "encryptionkey@0"
I20260806 13:10:56.098640 14331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:56.101686 14331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:56.103406 14331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:56.103525 14331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:56.107192 14331 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:35221
I20260806 13:10:56.107224 14420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:35221 every 8 connection(s)
I20260806 13:10:56.107486 14331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:56.108368 14421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:56.110415 14421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239: Bootstrap starting.
I20260806 13:10:56.110920 14421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:56.111160 14421 log.cc:824] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:56.111744 14421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239: No bootstrap required, opened a new log
I20260806 13:10:56.113212 14421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057763197c4c47b8854d33e415049239" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35221 } }
I20260806 13:10:56.113339 14421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 13:10:56.113370 14421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 057763197c4c47b8854d33e415049239, State: Initialized, Role: FOLLOWER
I20260806 13:10:56.113476 14421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057763197c4c47b8854d33e415049239" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35221 } }
I20260806 13:10:56.113557 14421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 13:10:56.113590 14421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 13:10:56.113626 14421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:56.114452 14421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057763197c4c47b8854d33e415049239" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35221 } }
I20260806 13:10:56.114542 14421 leader_election.cc:302] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 057763197c4c47b8854d33e415049239; no voters:
I20260806 13:10:56.114676 14421 leader_election.cc:288] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 13:10:56.114713 14425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:56.114872 14425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [term 1 LEADER]: Becoming Leader. State: Replica: 057763197c4c47b8854d33e415049239, State: Running, Role: LEADER
I20260806 13:10:56.114905 14421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:56.114957 14425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057763197c4c47b8854d33e415049239" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35221 } }
I20260806 13:10:56.115301 14427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "057763197c4c47b8854d33e415049239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057763197c4c47b8854d33e415049239" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35221 } } }
I20260806 13:10:56.115327 14428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 057763197c4c47b8854d33e415049239. Latest consensus state: current_term: 1 leader_uuid: "057763197c4c47b8854d33e415049239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057763197c4c47b8854d33e415049239" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 35221 } } }
I20260806 13:10:56.115411 14427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:56.115429 14428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:56.115621 14437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:56.116063 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 14331
I20260806 13:10:56.116189 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:56.116365 14209 external_mini_cluster.cc:1428] Setting key 6e8a73f9df4e244267adce97853da978
I20260806 13:10:56.116710 14437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:56.118438 14437 catalog_manager.cc:1347] Generated new cluster ID: 02bc8bc9b89c49f6bd685b4e261673e4
I20260806 13:10:56.118491 14437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:56.118891 14451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:56.116987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50449 (local address 127.13.224.126:35221)
0806 13:10:56.117196 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:56.117204 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:56.117454 (+ 250us) server_negotiation.cc:407] Connection header received
0806 13:10:56.117564 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:56.117571 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:56.117601 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:56.117736 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:56.118420 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:56.118423 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:56.118423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:56.118427 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:56.118469 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:56.118483 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:56.118536 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:56.118693 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 13:10:56.118728 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":102,"threads_started":1}
I20260806 13:10:56.119738 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:56.122231 14451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:56.121221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60791 (local address 127.13.224.126:35221)
0806 13:10:56.121256 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:56.121258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:56.121385 (+ 127us) server_negotiation.cc:407] Connection header received
0806 13:10:56.121453 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:56.121454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:56.121459 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:56.121501 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:56.122011 (+ 510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:56.122012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:56.122039 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 13:10:56.122043 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:56.122054 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 13:10:56.122058 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:56.122103 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:56.122164 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 13:10:56.122175 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:10:56.152776 14437 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:56.152912 14437 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:56.154834 14437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 057763197c4c47b8854d33e415049239: Generated new TSK 0
I20260806 13:10:56.154989 14437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:56.157740 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 14331
2026-08-06T13:10:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1380 ms)
[----------] 18 tests from AuthTokenIssuingTest (24655 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:56 dist-test-slave-q9h9 krb5kdc[14468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:56 dist-test-slave-q9h9 krb5kdc[14468](info): set up 2 sockets
Aug 06 13:10:56 dist-test-slave-q9h9 krb5kdc[14468](info): commencing operation
krb5kdc: starting...
W20260806 13:10:57.262567 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:10:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:10:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:10:57.368180 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:46115
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:46115,127.13.224.125:43523,127.13.224.124:42709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:10:57.445946 14492 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 13:10:57.446161 14492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:57.446185 14492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:57.446219 14492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:57.447345 14492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:57.447383 14492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:57.447403 14492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:57.447418 14492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:57.447434 14492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:57.448884 14492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:46115,127.13.224.125:43523,127.13.224.124:42709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:46115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:57.449111 14492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:57.449307 14492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:57.453836 14497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:57.453823 14498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:57.453933 14504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:57.453980 14492 server_base.cc:1034] running on GCE node
I20260806 13:10:57.454275 14492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:57.454519 14492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:57.455654 14492 hybrid_clock.cc:648] HybridClock initialized: now 1786021857455645 us; error 26 us; skew 500 ppm
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:57.457816 14492 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:10:57.458252 14492 webserver.cc:460] Webserver started at http://127.13.224.126:36987/ using document root <none> and password file <none>
I20260806 13:10:57.458451 14492 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:57.458499 14492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:57.458617 14492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:57.459573 14492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "a7ba228653b74147ad59046039314c53"
format_stamp: "Formatted at 2026-08-06 13:10:57 on dist-test-slave-q9h9"
server_key: "f0cce616e995d2a05346c2af39cac1f3"
server_key_iv: "eb36a0ede375affd4341fb7a448f838b"
server_key_version: "encryptionkey@0"
I20260806 13:10:57.459883 14492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "a7ba228653b74147ad59046039314c53"
format_stamp: "Formatted at 2026-08-06 13:10:57 on dist-test-slave-q9h9"
server_key: "f0cce616e995d2a05346c2af39cac1f3"
server_key_iv: "eb36a0ede375affd4341fb7a448f838b"
server_key_version: "encryptionkey@0"
I20260806 13:10:57.461025 14492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:10:57.461658 14516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:57.461805 14492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:10:57.461884 14492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "a7ba228653b74147ad59046039314c53"
format_stamp: "Formatted at 2026-08-06 13:10:57 on dist-test-slave-q9h9"
server_key: "f0cce616e995d2a05346c2af39cac1f3"
server_key_iv: "eb36a0ede375affd4341fb7a448f838b"
server_key_version: "encryptionkey@0"
I20260806 13:10:57.461946 14492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:57.475103 14492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:57.476557 14492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:57.476688 14492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:57.480284 14492 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:46115
I20260806 13:10:57.480320 14595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:46115 every 8 connection(s)
I20260806 13:10:57.480589 14492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:10:57.481324 14596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:57.481985 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 14492
I20260806 13:10:57.482128 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:10:57.482322 14209 external_mini_cluster.cc:1428] Setting key dae6cc3cc3bff88a796ce88513e0ebd9
I20260806 13:10:57.482977 14596 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.224.125@KRBTEST.COM; defaulting to no policy
I20260806 13:10:57.492723 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.483639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54562 (local address 127.13.224.126:46115)
0806 13:10:57.483804 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.483808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.483863 (+ 55us) server_negotiation.cc:407] Connection header received
0806 13:10:57.484479 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.484481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.484498 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.484561 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.485201 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.486388 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.487198 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.487659 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.487786 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.489730 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.489739 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.489739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.489756 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.491312 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.491663 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.491666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.491666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.491756 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.491954 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.491955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.491955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.492137 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.492204 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.492386 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.492508 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
I20260806 13:10:57.492754 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.483600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:46115 (local address 127.0.0.1:54562)
0806 13:10:57.483750 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.483786 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:57.484324 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:57.484590 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:57.484593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:57.484897 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:57.485107 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.485113 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.486481 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.486481 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.487119 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.487121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.487733 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.487734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.487789 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.488216 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:57.488224 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:57.489642 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:57.491397 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.491404 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.491406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.491592 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.491817 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.491818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.491819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.491899 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.492198 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:57.492203 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:57.492268 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 13:10:57.492345 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 13:10:57.492561 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":100,"thread_start_us":68,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.125@KRBTEST.COM" created.
I20260806 13:10:57.493396 14596 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:10:57.493686 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.493576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:43523 (local address 127.0.0.1:45884)
0806 13:10:57.493614 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.493665 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 13:10:57.493819 14523 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.224.125:43523: connect: Connection refused (error 111)
I20260806 13:10:57.494238 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.494161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:43523 (local address 127.0.0.1:45898)
0806 13:10:57.494198 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.494221 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 13:10:57.494333 14596 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.125:43523: Network error: Client connection negotiation failed: client connection to 127.13.224.125:43523: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.125.keytab.
Entry for principal kudu/127.13.224.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.125.keytab.
I20260806 13:10:57.505119 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.125:43523
--webserver_interface=127.13.224.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.125.keytab
--principal=kudu/127.13.224.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:46115,127.13.224.125:43523,127.13.224.124:42709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:10:57.543535 14596 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } attempt: 1
I20260806 13:10:57.543926 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.543815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:43523 (local address 127.0.0.1:45906)
0806 13:10:57.543863 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.543898 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 13:10:57.544219 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.544139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:43523 (local address 127.0.0.1:45914)
0806 13:10:57.544177 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.544202 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 13:10:57.544305 14596 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.125:43523: Network error: Client connection negotiation failed: client connection to 127.13.224.125:43523: connect: Connection refused (error 111)
W20260806 13:10:57.582870 14603 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 13:10:57.583043 14603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:57.583058 14603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:57.583077 14603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:57.584190 14603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:57.584235 14603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:57.584263 14603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:57.584280 14603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:57.584295 14603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:57.585757 14603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:46115,127.13.224.125:43523,127.13.224.124:42709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.125.keytab
--principal=kudu/127.13.224.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.125:43523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.224.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:57.585961 14603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:57.586174 14603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:57.590436 14612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:57.590447 14610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:57.590448 14614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:57.590687 14603 server_base.cc:1034] running on GCE node
I20260806 13:10:57.590905 14603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:57.591190 14603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:57.592327 14603 hybrid_clock.cc:648] HybridClock initialized: now 1786021857592318 us; error 26 us; skew 500 ppm
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:57.594581 14603 init.cc:377] Logged in from keytab as kudu/127.13.224.125@KRBTEST.COM (short username kudu)
I20260806 13:10:57.594980 14603 webserver.cc:460] Webserver started at http://127.13.224.125:44335/ using document root <none> and password file <none>
I20260806 13:10:57.595182 14603 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:57.595228 14603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:57.595356 14603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:57.596405 14603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/data/instance:
uuid: "e2b1b8f2619c405cacee517ef1aad84d"
format_stamp: "Formatted at 2026-08-06 13:10:57 on dist-test-slave-q9h9"
server_key: "d3114d81329b4423e80dc301575b256c"
server_key_iv: "58a69e9d68ac28cce2d4870a8822095a"
server_key_version: "encryptionkey@0"
I20260806 13:10:57.596746 14603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/wal/instance:
uuid: "e2b1b8f2619c405cacee517ef1aad84d"
format_stamp: "Formatted at 2026-08-06 13:10:57 on dist-test-slave-q9h9"
server_key: "d3114d81329b4423e80dc301575b256c"
server_key_iv: "58a69e9d68ac28cce2d4870a8822095a"
server_key_version: "encryptionkey@0"
I20260806 13:10:57.597976 14603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 13:10:57.598685 14623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:57.598845 14603 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 13:10:57.598922 14603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/wal
uuid: "e2b1b8f2619c405cacee517ef1aad84d"
format_stamp: "Formatted at 2026-08-06 13:10:57 on dist-test-slave-q9h9"
server_key: "d3114d81329b4423e80dc301575b256c"
server_key_iv: "58a69e9d68ac28cce2d4870a8822095a"
server_key_version: "encryptionkey@0"
I20260806 13:10:57.598975 14603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:57.612443 14596 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } attempt: 2
I20260806 13:10:57.612753 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.612632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:43523 (local address 127.0.0.1:45928)
0806 13:10:57.612682 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.612720 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 13:10:57.613046 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.612967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:43523 (local address 127.0.0.1:45932)
0806 13:10:57.613004 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.613029 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:43523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 13:10:57.613126 14596 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.125:43523: Network error: Client connection negotiation failed: client connection to 127.13.224.125:43523: connect: Connection refused (error 111)
I20260806 13:10:57.616153 14603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:57.617697 14603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:57.617808 14603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:57.621662 14603 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.125:43523
I20260806 13:10:57.621690 14693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.125:43523 every 8 connection(s)
I20260806 13:10:57.622112 14603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/data/info.pb
I20260806 13:10:57.622874 14694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:57.624166 14694 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:10:57.629251 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 14603
I20260806 13:10:57.629402 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-1/wal/instance
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.125@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:57.629668 14209 external_mini_cluster.cc:1428] Setting key f93b67ab18b16e09c227e92b7d710f46
I20260806 13:10:57.633445 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.624753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54576 (local address 127.13.224.126:46115)
0806 13:10:57.624799 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.624801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.624989 (+ 188us) server_negotiation.cc:407] Connection header received
0806 13:10:57.625458 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.625459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.625471 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.625546 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.626304 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.626775 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.627678 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.627880 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.627935 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.630299 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.630301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.630301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.630310 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.631803 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.632263 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.632264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.632265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.632346 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.632661 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.632662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.632663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.632795 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.632864 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.633271 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.633358 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 13:10:57.633919 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.624713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:46115 (local address 127.0.0.1:54576)
0806 13:10:57.624890 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.624921 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:57.625354 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:57.625619 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:57.625623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:57.625921 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:57.626192 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.626198 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.626854 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.626855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.627598 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.627600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.627947 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.627948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.628068 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.628517 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:57.628525 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:57.630176 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:57.631884 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.631891 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.631893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.632176 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.632427 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.632428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.632429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.632564 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.632859 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:57.632867 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:57.633094 (+ 227us) client_negotiation.cc:769] Sending connection context
0806 13:10:57.633557 (+ 463us) client_negotiation.cc:240] Negotiation successful
0806 13:10:57.633715 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260806 13:10:57.634548 14694 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.125@KRBTEST.COM for kudu/127.13.224.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.224.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.124@KRBTEST.COM" created.
I20260806 13:10:57.640954 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.634757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:43523 (local address 127.0.0.1:45934)
0806 13:10:57.634796 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.634815 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:57.634849 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:57.635086 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:57.635087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:57.635230 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:57.635282 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.635284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.636038 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.636039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.636790 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.636793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.637194 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.637196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.637269 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.637567 (+ 298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:57.637572 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:57.638714 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:57.640082 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.640083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.640083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.640231 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.640400 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.640400 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.640401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.640464 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.640663 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:57.640663 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:57.640694 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 13:10:57.640742 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 13:10:57.640884 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":61}
I20260806 13:10:57.641018 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.634836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45934 (local address 127.13.224.125:43523)
0806 13:10:57.634969 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.634972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.634986 (+ 14us) server_negotiation.cc:407] Connection header received
0806 13:10:57.635000 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.635001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.635010 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.635059 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.635347 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.635967 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.636863 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.637132 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.637174 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.638803 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.638810 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.638811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.638818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.640009 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.640292 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.640294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.640294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.640338 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.640511 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.640511 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.640512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.640617 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.640672 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.640761 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.640837 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":86,"threads_started":1}
I20260806 13:10:57.641503 14694 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:10:57.641773 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.641690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42709 (local address 127.0.0.1:50718)
0806 13:10:57.641721 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.641753 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 13:10:57.641885 14627 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
I20260806 13:10:57.642271 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.642186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42709 (local address 127.0.0.1:50726)
0806 13:10:57.642229 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.642254 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 13:10:57.642360 14694 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:42709: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.124.keytab.
Entry for principal kudu/127.13.224.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.124.keytab.
I20260806 13:10:57.653363 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.124:42709
--webserver_interface=127.13.224.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.124.keytab
--principal=kudu/127.13.224.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:46115,127.13.224.125:43523,127.13.224.124:42709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:10:57.691558 14694 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } attempt: 1
I20260806 13:10:57.691938 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.691797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42709 (local address 127.0.0.1:50738)
0806 13:10:57.691859 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.691903 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 13:10:57.692261 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.692171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42709 (local address 127.0.0.1:50750)
0806 13:10:57.692209 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.692239 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 13:10:57.692349 14694 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:42709: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
I20260806 13:10:57.704254 14596 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } attempt: 3
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for kudu/127.13.224.125@KRBTEST.COM
I20260806 13:10:57.711083 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.704577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45942 (local address 127.13.224.125:43523)
0806 13:10:57.704618 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.704620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.704649 (+ 29us) server_negotiation.cc:407] Connection header received
0806 13:10:57.704739 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.704740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.704755 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.704792 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.705163 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.705621 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.706689 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.706907 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.706958 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.708866 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.708867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.708868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.708874 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.710087 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.710460 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.710460 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.710461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.710504 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.710690 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.710691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.710691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.710784 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.710827 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.710977 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.711030 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 13:10:57.711162 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.704542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:43523 (local address 127.0.0.1:45942)
0806 13:10:57.704602 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.704611 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:57.704668 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:57.704819 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:57.704820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:57.705001 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:57.705096 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.705098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.705696 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.705697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.706595 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.706599 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.706972 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.706973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.707059 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.707449 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:57.707455 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:57.708782 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:57.710158 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.710159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.710160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.710379 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.710559 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.710560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.710560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.710633 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.710837 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:57.710838 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:57.710886 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 13:10:57.710957 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 13:10:57.711063 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 13:10:57.711536 14596 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:10:57.711853 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.711748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42709 (local address 127.0.0.1:50758)
0806 13:10:57.711794 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.711828 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 13:10:57.712200 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.712103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42709 (local address 127.0.0.1:50770)
0806 13:10:57.712147 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.712177 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 13:10:57.712292 14596 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:42709: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
W20260806 13:10:57.735260 14705 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 13:10:57.735435 14705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:10:57.735467 14705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:10:57.735505 14705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:10:57.736654 14705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:10:57.736698 14705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:10:57.736716 14705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:10:57.736743 14705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:10:57.736770 14705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:10:57.738229 14705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:46115,127.13.224.125:43523,127.13.224.124:42709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.124.keytab
--principal=kudu/127.13.224.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.124:42709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.224.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:10:57.738449 14705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:10:57.738667 14705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:10:57.743029 14713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:57.743126 14705 server_base.cc:1034] running on GCE node
W20260806 13:10:57.743042 14715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:10:57.743037 14711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:10:57.743427 14596 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } attempt: 1
I20260806 13:10:57.743476 14705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:10:57.743789 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.743667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42709 (local address 127.0.0.1:50774)
0806 13:10:57.743719 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.743760 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 13:10:57.743795 14705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:10:57.744086 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.744012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42709 (local address 127.0.0.1:50788)
0806 13:10:57.744041 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.744064 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260806 13:10:57.744184 14596 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:42709: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
I20260806 13:10:57.744943 14705 hybrid_clock.cc:648] HybridClock initialized: now 1786021857744927 us; error 30 us; skew 500 ppm
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:10:57.747171 14705 init.cc:377] Logged in from keytab as kudu/127.13.224.124@KRBTEST.COM (short username kudu)
I20260806 13:10:57.747615 14705 webserver.cc:460] Webserver started at http://127.13.224.124:37029/ using document root <none> and password file <none>
I20260806 13:10:57.747825 14705 fs_manager.cc:359] Metadata directory not provided
I20260806 13:10:57.747871 14705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:10:57.748003 14705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:10:57.749014 14705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/data/instance:
uuid: "066b55042ddf4ee2b8053ffcf4f9859d"
format_stamp: "Formatted at 2026-08-06 13:10:57 on dist-test-slave-q9h9"
server_key: "df9c72108ab6c5f5e8886a45b50d6a34"
server_key_iv: "948233da58023352206efa4637ab974d"
server_key_version: "encryptionkey@0"
I20260806 13:10:57.749357 14705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/wal/instance:
uuid: "066b55042ddf4ee2b8053ffcf4f9859d"
format_stamp: "Formatted at 2026-08-06 13:10:57 on dist-test-slave-q9h9"
server_key: "df9c72108ab6c5f5e8886a45b50d6a34"
server_key_iv: "948233da58023352206efa4637ab974d"
server_key_version: "encryptionkey@0"
I20260806 13:10:57.750659 14705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 13:10:57.751327 14726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:10:57.751470 14705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:10:57.751540 14705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/wal
uuid: "066b55042ddf4ee2b8053ffcf4f9859d"
format_stamp: "Formatted at 2026-08-06 13:10:57 on dist-test-slave-q9h9"
server_key: "df9c72108ab6c5f5e8886a45b50d6a34"
server_key_iv: "948233da58023352206efa4637ab974d"
server_key_version: "encryptionkey@0"
I20260806 13:10:57.751618 14705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:10:57.760483 14694 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } attempt: 2
I20260806 13:10:57.760821 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.760712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42709 (local address 127.0.0.1:50796)
0806 13:10:57.760764 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.760796 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 13:10:57.761106 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.761029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42709 (local address 127.0.0.1:50808)
0806 13:10:57.761066 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.761089 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 13:10:57.761181 14694 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:42709: Network error: Client connection negotiation failed: client connection to 127.13.224.124:42709: connect: Connection refused (error 111)
I20260806 13:10:57.763069 14705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:10:57.764736 14705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:10:57.764849 14705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:10:57.768724 14705 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.124:42709
I20260806 13:10:57.768759 14792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.124:42709 every 8 connection(s)
I20260806 13:10:57.769078 14705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/data/info.pb
I20260806 13:10:57.769877 14794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:10:57.771231 14794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.124@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:57.777377 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 14705
I20260806 13:10:57.777463 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786021591455662-14209-0/minicluster-data/master-2/wal/instance
I20260806 13:10:57.777631 14209 external_mini_cluster.cc:1428] Setting key f5b6583aa09cefdfc2a2406f9f27401e
I20260806 13:10:57.778977 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.771800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54592 (local address 127.13.224.126:46115)
0806 13:10:57.771876 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.771878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.772022 (+ 144us) server_negotiation.cc:407] Connection header received
0806 13:10:57.772429 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.772429 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.772439 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.772495 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.773011 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.773472 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.774395 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.774582 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.774635 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.776434 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.776435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.776436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.776442 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.777553 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.778061 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.778062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.778063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.778126 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.778395 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.778395 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.778396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.778529 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.778597 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.778820 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.778896 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 13:10:57.779038 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.771770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:46115 (local address 127.0.0.1:54592)
0806 13:10:57.771922 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.771953 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:57.772358 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:57.772543 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:57.772546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:57.772770 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:57.772939 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.772943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.773535 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.773536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.774328 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.774330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.774645 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.774646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.774731 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.775147 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:57.775154 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:57.776348 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:57.777630 (+ 1282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.777638 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.777640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.777924 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.778190 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.778191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.778191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.778323 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.778594 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:57.778598 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:57.778699 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 13:10:57.778783 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 13:10:57.778893 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260806 13:10:57.779520 14794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.124@KRBTEST.COM
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.124@KRBTEST.COM for kudu/127.13.224.125@KRBTEST.COM
I20260806 13:10:57.786267 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.777978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50810 (local address 127.13.224.124:42709)
0806 13:10:57.778134 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.778138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.778266 (+ 128us) server_negotiation.cc:407] Connection header received
0806 13:10:57.778308 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.778311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.778324 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.778401 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.779160 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.779999 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.780963 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.781396 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.781476 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.783605 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.783613 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.783613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.783621 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.785087 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.785444 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.785447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.785447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.785500 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.785698 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.785699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.785699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.785856 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.786004 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.786101 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.786175 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":13,"server-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260806 13:10:57.786593 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.779780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45944 (local address 127.13.224.125:43523)
0806 13:10:57.779818 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.779819 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.779828 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:10:57.779877 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.779878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.779885 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.779924 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.780202 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.780891 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.781727 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.782001 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.782082 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.784059 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.784060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.784061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.784067 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.785321 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.785670 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.785671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.785672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.785724 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.785917 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.785917 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.785917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.786053 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.786105 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.786443 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.786516 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:10:57.786820 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.779717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:43523 (local address 127.0.0.1:45944)
0806 13:10:57.779763 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.779774 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:57.779819 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:57.779931 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:57.779931 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:57.780083 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:57.780133 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.780134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.780963 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.780964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.781642 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.781645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.782099 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.782099 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.782151 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.782475 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:57.782480 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:57.783976 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:57.785392 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.785392 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.785393 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.785595 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.785798 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.785798 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.785799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.785857 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.786136 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:57.786137 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:57.786180 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 13:10:57.786241 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 13:10:57.786730 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":1}
I20260806 13:10:57.787110 14794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.124@KRBTEST.COM for kudu/127.13.224.124@KRBTEST.COM
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:10:57.793143 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.787365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42709 (local address 127.0.0.1:50812)
0806 13:10:57.787418 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.787425 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:57.787458 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:57.787592 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:57.787592 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:57.787715 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:57.787745 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.787747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.788376 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.788377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.788822 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.788824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.789156 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.789157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.789200 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.789520 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:57.789523 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:57.790643 (+ 1120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:57.792184 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.792185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.792185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.792344 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.792529 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.792529 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.792530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.792579 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.792797 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:57.792798 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:57.792840 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 13:10:57.792907 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 13:10:57.792990 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 13:10:57.793205 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.787391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50812 (local address 127.13.224.124:42709)
0806 13:10:57.787423 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.787425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.787453 (+ 28us) server_negotiation.cc:407] Connection header received
0806 13:10:57.787511 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.787512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.787521 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.787579 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.787819 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.788298 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.788891 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.789096 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.789153 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.790733 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.790734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.790734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.790740 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.792100 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.792409 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.792410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.792411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.792464 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.792629 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.792629 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.792629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.792731 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.792782 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.792942 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.793050 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":15}
I20260806 13:10:57.793910 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.787106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54604 (local address 127.13.224.126:46115)
0806 13:10:57.787137 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.787138 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.787146 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:10:57.787192 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.787193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.787201 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.787243 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.788032 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.788457 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.789056 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.789227 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.789283 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.791349 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.791350 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.791350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.791357 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.792575 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.792911 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.792912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.792913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.792967 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.793166 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.793167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.793167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.793580 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.793703 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.793804 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.793857 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:10:57.794574 14794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d: Bootstrap starting.
I20260806 13:10:57.795498 14794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:57.795830 14794 log.cc:824] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:57.796563 14794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d: No bootstrap required, opened a new log
I20260806 13:10:57.797909 14794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } }
I20260806 13:10:57.798079 14794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 066b55042ddf4ee2b8053ffcf4f9859d, State: Initialized, Role: FOLLOWER
I20260806 13:10:57.798177 14794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } }
I20260806 13:10:57.798454 14800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } } }
I20260806 13:10:57.798497 14794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:57.798547 14800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.125@KRBTEST.COM
W20260806 13:10:57.800238 14818 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:10:57.800333 14818 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:10:57.800601 14705 master_runner.cc:186] Error getting master registration for 127.13.224.126:46115: OK, instance_id { permanent_uuid: "a7ba228653b74147ad59046039314c53" instance_seqno: 1786021857479295 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 13:10:57.802197 14705 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 13:10:57.802467 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.794653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45952 (local address 127.13.224.125:43523)
0806 13:10:57.794690 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.794692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.794700 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:10:57.794720 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.794721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.794730 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.794815 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.795188 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.795808 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.796590 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.796845 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.796901 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.799074 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.799076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.799076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.799083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.800761 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.801177 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.801178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.801178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.801263 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.801508 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.801509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.801509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.801641 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.801805 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.802272 (+ 467us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.802349 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:10:57.803726 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:10:57.804816 14556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a7ba228653b74147ad59046039314c53"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54604
I20260806 13:10:57.819293 14596 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } attempt: 2
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for kudu/127.13.224.124@KRBTEST.COM
I20260806 13:10:57.825582 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.819544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50824 (local address 127.13.224.124:42709)
0806 13:10:57.819586 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.819588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.819594 (+ 6us) server_negotiation.cc:407] Connection header received
0806 13:10:57.819670 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.819671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.819678 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.819710 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.819995 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.820401 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.821286 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.821512 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.821567 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.823473 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.823475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.823475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.823481 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.824632 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.824924 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.824925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.824926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.824974 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.825125 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.825126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.825126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.825242 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.825303 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.825435 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.825497 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 13:10:57.825585 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.819489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42709 (local address 127.0.0.1:50824)
0806 13:10:57.819543 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.819556 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:57.819605 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:57.819733 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:57.819734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:57.819895 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:57.819936 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.819938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.820446 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.820447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.821209 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.821211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.821577 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.821578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.821631 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.822005 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:57.822010 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:57.823389 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:57.824706 (+ 1317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.824706 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.824707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.824866 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.825025 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.825026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.825026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.825078 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.825306 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:57.825307 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:57.825350 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 13:10:57.825409 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 13:10:57.825497 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 13:10:57.826812 14596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53: Bootstrap starting.
I20260806 13:10:57.827369 14596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:57.827620 14596 log.cc:824] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:57.828327 14596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53: No bootstrap required, opened a new log
I20260806 13:10:57.829229 14596 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } }
I20260806 13:10:57.829339 14596 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ba228653b74147ad59046039314c53, State: Initialized, Role: FOLLOWER
I20260806 13:10:57.829396 14596 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } }
I20260806 13:10:57.829689 14596 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:57.829658 14826 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } } }
I20260806 13:10:57.829731 14826 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:10:57.829747 14556 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 13:10:57.829820 14556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:57.830608 14556 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } }
I20260806 13:10:57.831161 14757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7ba228653b74147ad59046039314c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "066b55042ddf4ee2b8053ffcf4f9859d"
I20260806 13:10:57.831278 14757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d [term 0 FOLLOWER]: Advancing to term 1
W20260806 13:10:57.831466 14839 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:10:57.831511 14839 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:10:57.830847 14556 leader_election.cc:288] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [CANDIDATE]: Term 1 election: Requested vote from peers e2b1b8f2619c405cacee517ef1aad84d (127.13.224.125:43523), 066b55042ddf4ee2b8053ffcf4f9859d (127.13.224.124:42709)
I20260806 13:10:57.832127 14757 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7ba228653b74147ad59046039314c53 in term 1.
I20260806 13:10:57.832767 14520 leader_election.cc:302] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 066b55042ddf4ee2b8053ffcf4f9859d, a7ba228653b74147ad59046039314c53; no voters:
I20260806 13:10:57.832894 14826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:10:57.833022 14826 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [term 1 LEADER]: Becoming Leader. State: Replica: a7ba228653b74147ad59046039314c53, State: Running, Role: LEADER
I20260806 13:10:57.833106 14826 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } }
I20260806 13:10:57.833566 14847 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7ba228653b74147ad59046039314c53. Latest consensus state: current_term: 1 leader_uuid: "a7ba228653b74147ad59046039314c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } } }
I20260806 13:10:57.833664 14847 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:57.833932 14850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:10:57.834267 14850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 13:10:57.833148 14492 master_runner.cc:186] Error getting master registration for 127.13.224.125:43523: OK, instance_id { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" instance_seqno: 1786021857620590 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 13:10:57.835332 14492 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 13:10:57.837904 14757 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d [term 1 FOLLOWER]: Refusing update from remote peer a7ba228653b74147ad59046039314c53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:57.838371 14847 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:57.839574 14800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7ba228653b74147ad59046039314c53. Latest consensus state: current_term: 1 leader_uuid: "a7ba228653b74147ad59046039314c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } } }
I20260806 13:10:57.839653 14800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:10:57.841203 14800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7ba228653b74147ad59046039314c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } } }
I20260806 13:10:57.841274 14800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:10:57.841262 14826 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7ba228653b74147ad59046039314c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } } }
I20260806 13:10:57.841370 14826 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:57.841845 14852 mvcc.cc:205] Tried to move back new op lower bound from 7315545529701216256 to 7315545529685069824. Current Snapshot: MvccSnapshot[applied={T|T < 7315545529701216256}]
I20260806 13:10:57.842582 14853 mvcc.cc:205] Tried to move back new op lower bound from 7315545529701216256 to 7315545529685069824. Current Snapshot: MvccSnapshot[applied={T|T < 7315545529701216256}]
I20260806 13:10:57.842861 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.832777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.13.224.126:46115)
0806 13:10:57.832809 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.832811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.833510 (+ 699us) server_negotiation.cc:407] Connection header received
0806 13:10:57.833919 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.833920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.833930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.833961 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.834771 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.835456 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.836512 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.836822 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.836886 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.839110 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.839112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.839112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.839119 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.840908 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.841966 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.841967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.841968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.842044 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.842281 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.842282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.842283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.842402 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.842486 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.842702 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.842777 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:10:57.843482 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.833093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38462 (local address 127.13.224.125:43523)
0806 13:10:57.833149 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.833151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.836274 (+ 3123us) server_negotiation.cc:407] Connection header received
0806 13:10:57.836328 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.836329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.836351 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.836404 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.836997 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.837600 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.838857 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.839124 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.839185 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.840449 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.840450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.840450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.840457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.841984 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.842402 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.842403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.842403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.842462 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.842656 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.842657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.842657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.842774 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.842859 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.843318 (+ 459us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.843394 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 13:10:57.844753 14850 catalog_manager.cc:1347] Generated new cluster ID: 1a5fba963de546fdb2f0f97ef980bbc9
I20260806 13:10:57.844839 14850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:10:57.851980 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.835559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33270 (local address 127.13.224.124:42709)
0806 13:10:57.835590 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.835592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.843508 (+ 7916us) server_negotiation.cc:407] Connection header received
0806 13:10:57.843636 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.843637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.843648 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.843680 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.844422 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.845065 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.845991 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.846288 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.846348 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.848536 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.848537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.848538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.848544 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.850264 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.850882 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.850883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.850883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.850937 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.851243 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.851243 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.851244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.851384 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.851459 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.851824 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.851898 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:10:57.852289 14694 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } attempt: 3
Aug 06 13:10:57 dist-test-slave-q9h9 krb5kdc[14468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021857, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.125@KRBTEST.COM for kudu/127.13.224.124@KRBTEST.COM
I20260806 13:10:57.860796 14850 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:10:57.861522 14850 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:10:57.863520 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.852606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:42709 (local address 127.0.0.1:33282)
0806 13:10:57.852648 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 13:10:57.852657 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:10:57.852689 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:10:57.853208 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:10:57.853209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:10:57.853380 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:10:57.853417 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.853419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.855109 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.855110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.855875 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:10:57.855877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.856672 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.856673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:10:57.856743 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.857169 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:10:57.857191 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:10:57.859398 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:10:57.862190 (+ 2792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.862191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.862191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.862401 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.862727 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:10:57.862727 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:10:57.862728 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:10:57.862786 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:10:57.863214 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:10:57.863215 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:10:57.863282 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 13:10:57.863338 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 13:10:57.863427 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 13:10:57.864056 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.852861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33282 (local address 127.13.224.124:42709)
0806 13:10:57.853065 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.853067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.853074 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:10:57.853104 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.853104 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.853112 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.853157 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.853506 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.854274 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.855999 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.856534 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.859478 (+ 2944us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.859549 (+ 71us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:10:57.859550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:10:57.859550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:10:57.859555 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:10:57.862064 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.862497 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.862530 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.862530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.862613 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:10:57.862863 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:10:57.862863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:10:57.862864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:10:57.863069 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:10:57.863683 (+ 614us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.863758 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.863887 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
I20260806 13:10:57.865448 14694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d: Bootstrap starting.
I20260806 13:10:57.866906 14694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d: Neither blocks nor log segments found. Creating new log.
I20260806 13:10:57.867211 14694 log.cc:824] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d: Log is configured to *not* fsync() on all Append() calls
I20260806 13:10:57.869547 14850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53: Generated new TSK 0
I20260806 13:10:57.869813 14850 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 13:10:57.874647 14694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d: No bootstrap required, opened a new log
I20260806 13:10:57.876607 14694 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } }
I20260806 13:10:57.876739 14694 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b1b8f2619c405cacee517ef1aad84d, State: Initialized, Role: FOLLOWER
I20260806 13:10:57.877389 14694 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } }
I20260806 13:10:57.878010 14660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:10:57.878713 14661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7ba228653b74147ad59046039314c53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e2b1b8f2619c405cacee517ef1aad84d"
I20260806 13:10:57.878856 14660 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d [term 1 FOLLOWER]: Refusing update from remote peer a7ba228653b74147ad59046039314c53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:10:57.879179 14857 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:10:57.877925 14694 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:10:57.882434 14661 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7ba228653b74147ad59046039314c53 in term 1.
I20260806 13:10:57.883862 14864 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7ba228653b74147ad59046039314c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } } }
I20260806 13:10:57.883982 14864 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:10:57.884034 14864 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7ba228653b74147ad59046039314c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } } }
I20260806 13:10:57.884073 14864 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:10:57.884294 14861 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7ba228653b74147ad59046039314c53. Latest consensus state: current_term: 1 leader_uuid: "a7ba228653b74147ad59046039314c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } } }
I20260806 13:10:57.884356 14861 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:10:57.885936 14885 catalog_manager.cc:1259] Loaded cluster ID: 1a5fba963de546fdb2f0f97ef980bbc9
I20260806 13:10:57.885980 14885 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d: loading cluster ID for follower catalog manager: success
I20260806 13:10:57.887048 14885 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d: acquiring CA information for follower catalog manager: success
I20260806 13:10:57.887425 14885 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e2b1b8f2619c405cacee517ef1aad84d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 13:10:57.906658 14797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:57.904964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33284 (local address 127.13.224.124:42709)
0806 13:10:57.905007 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.905009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.905941 (+ 932us) server_negotiation.cc:407] Connection header received
0806 13:10:57.906087 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.906088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.906100 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.906130 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.906597 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33284: BlockingRecv error: recv got EOF from 127.0.0.1:33284 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 13:10:57.907404 14599 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:57.905187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34224 (local address 127.13.224.126:46115)
0806 13:10:57.905227 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.905229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.906894 (+ 1665us) server_negotiation.cc:407] Connection header received
0806 13:10:57.906986 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.906987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.906997 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.907025 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.907346 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34224: BlockingRecv error: recv got EOF from 127.0.0.1:34224 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 13:10:57.908162 14698 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:57.905678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38464 (local address 127.13.224.125:43523)
0806 13:10:57.905716 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.905718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.907676 (+ 1958us) server_negotiation.cc:407] Connection header received
0806 13:10:57.907770 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.907771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.907781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.907809 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.908110 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38464: BlockingRecv error: recv got EOF from 127.0.0.1:38464 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:10:57.913268 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.909803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38468 (local address 127.13.224.125:43523)
0806 13:10:57.909834 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.909836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.910063 (+ 227us) server_negotiation.cc:407] Connection header received
0806 13:10:57.910170 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.910171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.910204 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.910258 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:57.910466 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.911201 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.912327 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.912613 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.912656 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.912891 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:57.913005 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:57.913056 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.913177 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.913197 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 13:10:57.916446 14797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:57.915755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33300 (local address 127.13.224.124:42709)
0806 13:10:57.915798 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.915800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.915930 (+ 130us) server_negotiation.cc:407] Connection header received
0806 13:10:57.916028 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.916029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.916046 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.916074 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.916401 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33300: BlockingRecv error: recv got EOF from 127.0.0.1:33300 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 13:10:57.921757 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.918703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38476 (local address 127.13.224.125:43523)
0806 13:10:57.918736 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.918737 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.918969 (+ 232us) server_negotiation.cc:407] Connection header received
0806 13:10:57.919058 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.919059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.919068 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.919093 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:57.919263 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.919924 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.920991 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.921288 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.921332 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.921530 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:57.921560 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:57.921587 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.921685 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.921703 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 13:10:57.924676 14797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:57.923976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33310 (local address 127.13.224.124:42709)
0806 13:10:57.924008 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.924009 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.924208 (+ 199us) server_negotiation.cc:407] Connection header received
0806 13:10:57.924315 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.924316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.924327 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.924354 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.924637 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33310: BlockingRecv error: recv got EOF from 127.0.0.1:33310 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:10:57.930737 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.927718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38482 (local address 127.13.224.125:43523)
0806 13:10:57.927748 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.927749 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.927977 (+ 228us) server_negotiation.cc:407] Connection header received
0806 13:10:57.928065 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.928066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.928074 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.928100 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:57.928263 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.928927 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.929951 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.930243 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.930311 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.930509 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:57.930539 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:57.930564 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.930661 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.930680 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 13:10:57.933478 14797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:57.932787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33318 (local address 127.13.224.124:42709)
0806 13:10:57.932818 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.932820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.933016 (+ 196us) server_negotiation.cc:407] Connection header received
0806 13:10:57.933105 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.933106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.933115 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.933141 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.933437 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33318: BlockingRecv error: recv got EOF from 127.0.0.1:33318 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 13:10:57.941578 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.938461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38494 (local address 127.13.224.125:43523)
0806 13:10:57.938494 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.938495 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.938702 (+ 207us) server_negotiation.cc:407] Connection header received
0806 13:10:57.938787 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.938788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.938796 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.938821 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:57.938963 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.939641 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.940850 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.941121 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.941156 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.941352 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:57.941382 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:57.941409 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.941502 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.941517 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 13:10:57.944330 14797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:57.943649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33324 (local address 127.13.224.124:42709)
0806 13:10:57.943683 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.943685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.943879 (+ 194us) server_negotiation.cc:407] Connection header received
0806 13:10:57.943964 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.943965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.943974 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.944000 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.944290 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33324: BlockingRecv error: recv got EOF from 127.0.0.1:33324 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:10:57.956561 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.953301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38502 (local address 127.13.224.125:43523)
0806 13:10:57.953333 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.953335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.953598 (+ 263us) server_negotiation.cc:407] Connection header received
0806 13:10:57.953719 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.953720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.953729 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.953754 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:57.953945 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.954687 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.955810 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.956087 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.956126 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.956332 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:57.956361 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:57.956387 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.956485 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.956501 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 13:10:57.959389 14797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:57.958677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33340 (local address 127.13.224.124:42709)
0806 13:10:57.958710 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.958711 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.958909 (+ 198us) server_negotiation.cc:407] Connection header received
0806 13:10:57.958999 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.959000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.959010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.959038 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.959347 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33340: BlockingRecv error: recv got EOF from 127.0.0.1:33340 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 13:10:57.978822 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:57.976453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38504 (local address 127.13.224.125:43523)
0806 13:10:57.976499 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.976501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.976717 (+ 216us) server_negotiation.cc:407] Connection header received
0806 13:10:57.976780 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.976781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.976791 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.976835 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:57.976937 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.977495 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.978244 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:57.978438 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:57.978490 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:57.978615 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:57.978646 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:57.978689 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:57.978742 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 13:10:57.978759 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 13:10:57.980867 14797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:57.980319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33350 (local address 127.13.224.124:42709)
0806 13:10:57.980370 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:57.980372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:57.980469 (+ 97us) server_negotiation.cc:407] Connection header received
0806 13:10:57.980507 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:57.980508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:57.980517 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:57.980556 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:57.980825 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33350: BlockingRecv error: recv got EOF from 127.0.0.1:33350 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 13:10:58.016847 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:58.013850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38508 (local address 127.13.224.125:43523)
0806 13:10:58.013892 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:58.013893 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:58.014042 (+ 149us) server_negotiation.cc:407] Connection header received
0806 13:10:58.014102 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:58.014103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:58.014114 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:58.014177 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:58.014321 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:58.015044 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:58.016095 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:58.016379 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:58.016455 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:58.016613 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:58.016647 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:58.016697 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:58.016759 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 13:10:58.016777 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 13:10:58.022646 14797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:58.018734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.13.224.124:42709)
0806 13:10:58.018772 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:58.018774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:58.022173 (+ 3399us) server_negotiation.cc:407] Connection header received
0806 13:10:58.022230 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:58.022231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:58.022243 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:58.022304 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:58.022598 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33364: BlockingRecv error: recv got EOF from 127.0.0.1:33364 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 13:10:58.089696 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:58.087524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.13.224.125:43523)
0806 13:10:58.087589 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:58.087591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:58.087669 (+ 78us) server_negotiation.cc:407] Connection header received
0806 13:10:58.087734 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:58.087735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:58.087746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:58.087794 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:58.087910 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:58.088373 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:58.089102 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:58.089285 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:58.089339 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:58.089471 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:58.089501 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:58.089542 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:58.089594 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 13:10:58.089612 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 13:10:58.091638 14797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:58.091147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.13.224.124:42709)
0806 13:10:58.091222 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:58.091224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:58.091233 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:10:58.091288 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:58.091289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:58.091298 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:58.091341 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:58.091586 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33378: BlockingRecv error: recv got EOF from 127.0.0.1:33378 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 13:10:58.222841 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:58.220620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38530 (local address 127.13.224.125:43523)
0806 13:10:58.220673 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:58.220675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:58.220769 (+ 94us) server_negotiation.cc:407] Connection header received
0806 13:10:58.220842 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:58.220843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:58.220854 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:58.220896 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:58.221016 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:58.221482 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:58.222230 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:58.222415 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:58.222472 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:58.222603 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:58.222634 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:58.222674 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:58.222731 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 13:10:58.222762 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 13:10:58.224810 14797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:58.224237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33384 (local address 127.13.224.124:42709)
0806 13:10:58.224305 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:58.224307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:58.224336 (+ 29us) server_negotiation.cc:407] Connection header received
0806 13:10:58.224406 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:58.224407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:58.224418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:58.224464 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:58.224743 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33384: BlockingRecv error: recv got EOF from 127.0.0.1:33384 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 13:10:58.257758 14847 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7ba228653b74147ad59046039314c53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ba228653b74147ad59046039314c53" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 46115 } } peers { permanent_uuid: "e2b1b8f2619c405cacee517ef1aad84d" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 43523 } } peers { permanent_uuid: "066b55042ddf4ee2b8053ffcf4f9859d" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 42709 } } }
I20260806 13:10:58.257885 14847 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ba228653b74147ad59046039314c53 [sys.catalog]: This master's current role is: LEADER
I20260806 13:10:58.484143 14698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:58.481845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38538 (local address 127.13.224.125:43523)
0806 13:10:58.481905 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:58.481907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:58.481992 (+ 85us) server_negotiation.cc:407] Connection header received
0806 13:10:58.482078 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:58.482079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:58.482090 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:58.482137 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:58.482276 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:58.482754 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:58.483537 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:58.483733 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:58.483788 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:58.483927 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:58.483959 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:58.484000 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:58.484052 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 13:10:58.484072 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 13:10:58.486239 14797 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:10:58.485672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33388 (local address 127.13.224.124:42709)
0806 13:10:58.485722 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:58.485724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:58.485747 (+ 23us) server_negotiation.cc:407] Connection header received
0806 13:10:58.485804 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:58.485804 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:58.485816 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:58.485870 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:10:58.486189 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33388: BlockingRecv error: recv got EOF from 127.0.0.1:33388 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 13:10:58.800717 14818 catalog_manager.cc:1259] Loaded cluster ID: 1a5fba963de546fdb2f0f97ef980bbc9
I20260806 13:10:58.800786 14818 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d: loading cluster ID for follower catalog manager: success
I20260806 13:10:58.801540 14818 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d: acquiring CA information for follower catalog manager: success
I20260806 13:10:58.801797 14818 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 066b55042ddf4ee2b8053ffcf4f9859d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 13:10:59.001590 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:58.999231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38542 (local address 127.13.224.125:43523)
0806 13:10:58.999349 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:58.999352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:58.999394 (+ 42us) server_negotiation.cc:407] Connection header received
0806 13:10:58.999477 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:58.999478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:58.999490 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:58.999542 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:58.999661 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:59.000154 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:59.000895 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:59.001133 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:59.001191 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:59.001337 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:59.001375 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:59.001418 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:59.001474 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 13:10:59.001497 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":90,"threads_started":1}
I20260806 13:10:59.005513 15057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:59.003202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33392 (local address 127.13.224.124:42709)
0806 13:10:59.003365 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:59.003367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:59.003374 (+ 7us) server_negotiation.cc:407] Connection header received
0806 13:10:59.003398 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:59.003399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:59.003471 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:59.003541 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:59.003652 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:59.004117 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:59.004805 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:59.005007 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:59.005060 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:59.005193 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:59.005310 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:59.005376 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:59.005412 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 13:10:59.005434 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
I20260806 13:10:59.009658 15057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:59.006964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33404 (local address 127.13.224.124:42709)
0806 13:10:59.007001 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:59.007002 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:59.007212 (+ 210us) server_negotiation.cc:407] Connection header received
0806 13:10:59.007253 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:59.007254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:59.007262 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:59.007307 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:59.007445 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:59.008132 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:59.009050 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:59.009245 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:59.009296 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:59.009418 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:59.009446 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:59.009491 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:59.009565 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 13:10:59.009585 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:10:59.009851 15066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:59.007056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34234 (local address 127.13.224.126:46115)
0806 13:10:59.007174 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:59.007176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:59.007185 (+ 9us) server_negotiation.cc:407] Connection header received
0806 13:10:59.007230 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:59.007231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:59.007292 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:59.007365 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:59.007497 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:59.008224 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:59.008907 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:59.009200 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:59.009400 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:59.009454 (+ 54us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:59.009542 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:59.009727 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:59.009755 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 13:10:59.009776 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
I20260806 13:10:59.010140 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:10:59.007091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38548 (local address 127.13.224.125:43523)
0806 13:10:59.007169 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 13:10:59.007171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:10:59.007261 (+ 90us) server_negotiation.cc:407] Connection header received
0806 13:10:59.007329 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:10:59.007330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:10:59.007340 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:10:59.007366 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:10:59.007502 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:59.007959 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:59.008951 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:10:59.009234 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:10:59.009302 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:10:59.009767 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:10:59.009800 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:10:59.009846 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 13:10:59.009931 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 13:10:59.009956 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 13:10:59.011502 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 14492
I20260806 13:10:59.023160 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 14603
I20260806 13:10:59.031291 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 14705
2026-08-06T13:10:59Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2868 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 13:10:59 dist-test-slave-q9h9 krb5kdc[15069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 13:10:59 dist-test-slave-q9h9 krb5kdc[15069](info): set up 2 sockets
Aug 06 13:10:59 dist-test-slave-q9h9 krb5kdc[15069](info): commencing operation
krb5kdc: starting...
W20260806 13:11:00.362795 14209 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/test-user.keytab.
Aug 06 13:11:00 dist-test-slave-q9h9 krb5kdc[15069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T13:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T13:11:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.224.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
Entry for principal kudu/127.13.224.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab.
I20260806 13:11:00.532644 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:36291
--webserver_interface=127.13.224.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:37277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:36291,127.13.224.125:34169,127.13.224.124:40201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 13:11:00.665042 15088 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 13:11:00.665347 15088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:11:00.665388 15088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:11:00.665434 15088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:11:00.667093 15088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:11:00.667155 15088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:11:00.667196 15088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:11:00.667254 15088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:11:00.667306 15088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:11:00.669399 15088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:37277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:36291,127.13.224.125:34169,127.13.224.124:40201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.126.keytab
--principal=kudu/127.13.224.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.126:36291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.224.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:11:00.669792 15088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:11:00.670156 15088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:11:00.676421 15095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:11:00.676517 15096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:11:00.676430 15098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:11:00.676932 15088 server_base.cc:1034] running on GCE node
I20260806 13:11:00.677217 15088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:11:00.677580 15088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:11:00.678747 15088 hybrid_clock.cc:648] HybridClock initialized: now 1786021860678726 us; error 37 us; skew 500 ppm
Aug 06 13:11:00 dist-test-slave-q9h9 krb5kdc[15069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:11:00.682080 15088 init.cc:377] Logged in from keytab as kudu/127.13.224.126@KRBTEST.COM (short username kudu)
I20260806 13:11:00.682669 15088 webserver.cc:460] Webserver started at http://127.13.224.126:45941/ using document root <none> and password file <none>
I20260806 13:11:00.682861 15088 fs_manager.cc:359] Metadata directory not provided
I20260806 13:11:00.682906 15088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:11:00.683070 15088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:11:00.684463 15088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/data/instance:
uuid: "0145619affd84e1fa21f38a238637ac7"
format_stamp: "Formatted at 2026-08-06 13:11:00 on dist-test-slave-q9h9"
server_key: "aeb96380c696f3dbcf22c2c683bb8116"
server_key_iv: "c3a3285efb84b20665d8964c266e5e79"
server_key_version: "encryptionkey@0"
I20260806 13:11:00.685017 15088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/wal/instance:
uuid: "0145619affd84e1fa21f38a238637ac7"
format_stamp: "Formatted at 2026-08-06 13:11:00 on dist-test-slave-q9h9"
server_key: "aeb96380c696f3dbcf22c2c683bb8116"
server_key_iv: "c3a3285efb84b20665d8964c266e5e79"
server_key_version: "encryptionkey@0"
I20260806 13:11:00.687040 15088 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 13:11:00.688138 15107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:11:00.688366 15088 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:11:00.688499 15088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/wal
uuid: "0145619affd84e1fa21f38a238637ac7"
format_stamp: "Formatted at 2026-08-06 13:11:00 on dist-test-slave-q9h9"
server_key: "aeb96380c696f3dbcf22c2c683bb8116"
server_key_iv: "c3a3285efb84b20665d8964c266e5e79"
server_key_version: "encryptionkey@0"
I20260806 13:11:00.688587 15088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:11:00.708148 15088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:11:00.710754 15088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:11:00.710919 15088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:11:00.716452 15088 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.126:36291
I20260806 13:11:00.716500 15182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.126:36291 every 8 connection(s)
I20260806 13:11:00.716974 15088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/data/info.pb
I20260806 13:11:00.718215 15185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:11:00.718880 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 15088
I20260806 13:11:00.718981 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-0/wal/instance
I20260806 13:11:00.719203 14209 external_mini_cluster.cc:1428] Setting key 849349aaecbcd9f1e508e8eca991ab3c
I20260806 13:11:00.720443 15185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:11:00 dist-test-slave-q9h9 krb5kdc[15069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.224.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.125@KRBTEST.COM" created.
I20260806 13:11:00.733127 15190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.721333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.13.224.126:36291)
0806 13:11:00.721583 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:00.721594 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:00.721748 (+ 154us) server_negotiation.cc:407] Connection header received
0806 13:11:00.722534 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:00.722537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:00.722580 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:00.722746 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:00.723458 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.724433 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.725534 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.725954 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.726142 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:00.728712 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:00.728719 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:00.728719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:00.728726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:00.730818 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:00.731396 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:00.731399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:00.731399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:00.731527 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:00.731848 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:00.731849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:00.731850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:00.732144 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:00.732262 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:00.732526 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 13:11:00.732725 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":168,"thread_start_us":112,"threads_started":1}
I20260806 13:11:00.733151 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.721328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:36291 (local address 127.0.0.1:59370)
0806 13:11:00.721584 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.721634 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 13:11:00.722308 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:11:00.722734 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:11:00.722736 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:11:00.723079 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:11:00.723339 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:00.723344 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.724539 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.724540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:00.725434 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:00.725437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.726071 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.726072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:00.726179 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:00.726799 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:11:00.726812 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:11:00.728618 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:11:00.730930 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:00.730935 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:00.730942 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:00.731285 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:00.731619 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:00.731620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:00.731620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:00.731759 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:00.732246 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:11:00.732256 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:11:00.732346 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 13:11:00.732484 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 13:11:00.732818 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":112,"thread_start_us":117,"threads_started":1}
I20260806 13:11:00.734294 15185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:11:00.734742 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.734582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34169 (local address 127.0.0.1:53246)
0806 13:11:00.734638 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.734700 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:34169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 13:11:00.734912 15108 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.224.125:34169: connect: Connection refused (error 111)
I20260806 13:11:00.735431 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.735317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34169 (local address 127.0.0.1:53262)
0806 13:11:00.735366 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.735401 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:34169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 13:11:00.735569 15185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.125:34169: Network error: Client connection negotiation failed: client connection to 127.13.224.125:34169: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.125.keytab.
Entry for principal kudu/127.13.224.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.125.keytab.
I20260806 13:11:00.748150 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.125:34169
--webserver_interface=127.13.224.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.125.keytab
--principal=kudu/127.13.224.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:37277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:36291,127.13.224.125:34169,127.13.224.124:40201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:11:00.784842 15185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } attempt: 1
I20260806 13:11:00.785281 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.785164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34169 (local address 127.0.0.1:53276)
0806 13:11:00.785223 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.785253 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:34169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 13:11:00.785585 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.785505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34169 (local address 127.0.0.1:53282)
0806 13:11:00.785541 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.785565 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:34169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 13:11:00.785666 15185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.125:34169: Network error: Client connection negotiation failed: client connection to 127.13.224.125:34169: connect: Connection refused (error 111)
I20260806 13:11:00.853814 15185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } attempt: 2
I20260806 13:11:00.854431 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.854272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34169 (local address 127.0.0.1:53292)
0806 13:11:00.854341 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.854393 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:34169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 13:11:00.854965 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.854784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34169 (local address 127.0.0.1:53302)
0806 13:11:00.854861 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.854924 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.125:34169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 13:11:00.855134 15185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.125:34169: Network error: Client connection negotiation failed: client connection to 127.13.224.125:34169: connect: Connection refused (error 111)
W20260806 13:11:00.859567 15196 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 13:11:00.859903 15196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:11:00.859992 15196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:11:00.860067 15196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:11:00.861763 15196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:11:00.861828 15196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:11:00.861855 15196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:11:00.861914 15196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:11:00.861965 15196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:11:00.864459 15196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:37277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:36291,127.13.224.125:34169,127.13.224.124:40201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.125.keytab
--principal=kudu/127.13.224.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.125:34169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.224.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:11:00.864841 15196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:11:00.865461 15196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:11:00.874513 15203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:11:00.874527 15202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:11:00.874527 15205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:11:00.875135 15196 server_base.cc:1034] running on GCE node
I20260806 13:11:00.875326 15196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:11:00.875757 15196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:11:00.876950 15196 hybrid_clock.cc:648] HybridClock initialized: now 1786021860876903 us; error 66 us; skew 500 ppm
Aug 06 13:11:00 dist-test-slave-q9h9 krb5kdc[15069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:11:00.886778 15196 init.cc:377] Logged in from keytab as kudu/127.13.224.125@KRBTEST.COM (short username kudu)
I20260806 13:11:00.887720 15196 webserver.cc:460] Webserver started at http://127.13.224.125:43019/ using document root <none> and password file <none>
I20260806 13:11:00.888183 15196 fs_manager.cc:359] Metadata directory not provided
I20260806 13:11:00.888293 15196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:11:00.888492 15196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:11:00.891535 15196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/data/instance:
uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48"
format_stamp: "Formatted at 2026-08-06 13:11:00 on dist-test-slave-q9h9"
server_key: "9956e14908a046f4b4a2c0bb8d10372c"
server_key_iv: "c63a348dd68a01dc2a48e94c01d25288"
server_key_version: "encryptionkey@0"
I20260806 13:11:00.892504 15196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/wal/instance:
uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48"
format_stamp: "Formatted at 2026-08-06 13:11:00 on dist-test-slave-q9h9"
server_key: "9956e14908a046f4b4a2c0bb8d10372c"
server_key_iv: "c63a348dd68a01dc2a48e94c01d25288"
server_key_version: "encryptionkey@0"
I20260806 13:11:00.895846 15196 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 13:11:00.897254 15214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:11:00.897516 15196 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 13:11:00.897708 15196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/wal
uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48"
format_stamp: "Formatted at 2026-08-06 13:11:00 on dist-test-slave-q9h9"
server_key: "9956e14908a046f4b4a2c0bb8d10372c"
server_key_iv: "c63a348dd68a01dc2a48e94c01d25288"
server_key_version: "encryptionkey@0"
I20260806 13:11:00.897840 15196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:11:00.919924 15196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:11:00.922698 15196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:11:00.922930 15196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:11:00.930284 15196 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.125:34169
I20260806 13:11:00.930353 15288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.125:34169 every 8 connection(s)
I20260806 13:11:00.930984 15196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/data/info.pb
I20260806 13:11:00.932368 15289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:11:00.934360 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 15196
I20260806 13:11:00.934540 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-1/wal/instance
I20260806 13:11:00.934885 14209 external_mini_cluster.cc:1428] Setting key b37ccb63228a6cde9e88ea91a73a1d06
I20260806 13:11:00.934834 15289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 13:11:00 dist-test-slave-q9h9 krb5kdc[15069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.125@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:11:00.946288 15185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } attempt: 3
WARNING: no policy specified for kudu/127.13.224.124@KRBTEST.COM; defaulting to no policy
I20260806 13:11:00.949294 15190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.935825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59384 (local address 127.13.224.126:36291)
0806 13:11:00.935919 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:00.935921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:00.936220 (+ 299us) server_negotiation.cc:407] Connection header received
0806 13:11:00.936929 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:00.936931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:00.936960 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:00.937063 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:00.938456 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.939143 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.940569 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.941114 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.941311 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:00.945027 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:00.945029 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:00.945029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:00.945036 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:00.947381 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:00.948082 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:00.948084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:00.948084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:00.948168 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:00.948467 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:00.948468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:00.948468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:00.948647 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:00.948729 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:00.949092 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 13:11:00.949207 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":45}
I20260806 13:11:00.949632 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.935775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:36291 (local address 127.0.0.1:59384)
0806 13:11:00.936032 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.936115 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 13:11:00.936767 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:11:00.937142 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:11:00.937154 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:11:00.937686 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:11:00.938276 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:00.938294 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.939249 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.939250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:00.940414 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:00.940417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.941314 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.941315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:00.941485 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:00.942409 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:11:00.942424 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:11:00.944900 (+ 2476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:11:00.947537 (+ 2637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:00.947541 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:00.947543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:00.947980 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:00.948257 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:00.948258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:00.948259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:00.948390 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:00.948712 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:11:00.948716 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:11:00.948901 (+ 185us) client_negotiation.cc:769] Sending connection context
0806 13:11:00.949071 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 13:11:00.949318 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":105,"threads_started":1}
I20260806 13:11:00.950393 15289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.224.124@KRBTEST.COM" created.
Aug 06 13:11:00 dist-test-slave-q9h9 krb5kdc[15069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for kudu/127.13.224.125@KRBTEST.COM
Aug 06 13:11:00 dist-test-slave-q9h9 krb5kdc[15069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.125@KRBTEST.COM for kudu/127.13.224.125@KRBTEST.COM
I20260806 13:11:00.957283 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.946625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34169 (local address 127.0.0.1:53316)
0806 13:11:00.946694 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.946710 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 13:11:00.946776 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:11:00.947379 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:11:00.947380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:11:00.947574 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:11:00.947637 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:00.947639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.948983 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.948985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:00.950060 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:00.950068 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.950940 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.950941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:00.951052 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:00.951532 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:11:00.951536 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:11:00.953258 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:11:00.955892 (+ 2634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:00.955894 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:00.955894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:00.956117 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:00.956355 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:00.956356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:00.956356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:00.956432 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:00.956832 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:11:00.956833 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:11:00.956902 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 13:11:00.957013 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 13:11:00.957143 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 13:11:00.958279 15294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.946898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.13.224.125:34169)
0806 13:11:00.947154 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:00.947159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:00.947199 (+ 40us) server_negotiation.cc:407] Connection header received
0806 13:11:00.947230 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:00.947232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:00.947258 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:00.947366 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:00.947731 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.948882 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.950161 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.950799 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.951239 (+ 440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:00.953342 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:00.953349 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:00.953350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:00.953363 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:00.955808 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:00.956204 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:00.956209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:00.956209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:00.956288 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:00.956496 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:00.956498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:00.956498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:00.956714 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:00.956824 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:00.957695 (+ 871us) server_negotiation.cc:299] Negotiation successful
0806 13:11:00.957877 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":148,"threads_started":1}
I20260806 13:11:00.959074 15185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:11:00.959457 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.959339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40768)
0806 13:11:00.959395 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.959426 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 13:11:00.959818 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.959712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40784)
0806 13:11:00.959761 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.959789 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 13:11:00.959940 15185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:40201: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
I20260806 13:11:00.960868 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.950802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53330 (local address 127.13.224.125:34169)
0806 13:11:00.950979 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:00.950985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:00.950993 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:11:00.951016 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:00.951017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:00.951035 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:00.951137 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:00.952095 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.952759 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.953941 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.954363 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.954466 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:00.957016 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:00.957018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:00.957022 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:00.957029 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:00.959227 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:00.959859 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:00.959861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:00.959861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:00.959949 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:00.960255 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:00.960256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:00.960256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:00.960373 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:00.960441 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:00.960670 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 13:11:00.960753 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260806 13:11:00.961000 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.950697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34169 (local address 127.0.0.1:53330)
0806 13:11:00.950784 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.950795 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 13:11:00.950841 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:11:00.951171 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:11:00.951172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:11:00.951348 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:11:00.951415 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:00.951417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.952893 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.952899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:00.953782 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:00.953790 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:00.954450 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:00.954451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:00.954543 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:00.955006 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:11:00.955011 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:11:00.956908 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:11:00.959336 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:00.959337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:00.959338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:00.959766 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:00.960053 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:00.960054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:00.960054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:00.960147 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:00.960449 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:11:00.960451 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:11:00.960513 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 13:11:00.960642 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 13:11:00.960834 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":66}
I20260806 13:11:00.961665 15289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:11:00.962154 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.961922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40798)
0806 13:11:00.961982 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.962111 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 13:11:00.962364 15215 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
I20260806 13:11:00.962932 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.962776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40810)
0806 13:11:00.962835 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.962883 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 13:11:00.963132 15289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:40201: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.224.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.124.keytab.
Entry for principal kudu/127.13.224.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.124.keytab.
I20260806 13:11:00.969897 14209 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXc7v3G/build/release/bin/kudu
/tmp/dist-test-taskXc7v3G/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.124:40201
--webserver_interface=127.13.224.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.124.keytab
--principal=kudu/127.13.224.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.224.84:37277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.224.126:36291,127.13.224.125:34169,127.13.224.124:40201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 13:11:00.991091 15185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } attempt: 1
I20260806 13:11:00.991497 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.991371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40820)
0806 13:11:00.991434 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.991470 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 13:11:00.991918 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:00.991823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40832)
0806 13:11:00.991875 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:00.991899 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 13:11:00.992009 15185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:40201: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
I20260806 13:11:01.012379 15289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } attempt: 1
I20260806 13:11:01.012987 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.012844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40842)
0806 13:11:01.012908 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:01.012954 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 13:11:01.013522 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.013398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40846)
0806 13:11:01.013438 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:01.013480 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 13:11:01.013636 15289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:40201: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
I20260806 13:11:01.067157 15185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } attempt: 2
I20260806 13:11:01.067934 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.067677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40858)
0806 13:11:01.067783 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:01.067847 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 13:11:01.068998 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.068853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40870)
0806 13:11:01.068927 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:01.068950 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 13:11:01.069155 15185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:40201: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
I20260806 13:11:01.081782 15289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } attempt: 2
I20260806 13:11:01.082739 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.082474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40884)
0806 13:11:01.082595 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:01.082710 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 13:11:01.083700 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.083494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40898)
0806 13:11:01.083590 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:01.083658 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 13:11:01.083817 15289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.224.124:40201: Network error: Client connection negotiation failed: client connection to 127.13.224.124:40201: connect: Connection refused (error 111)
W20260806 13:11:01.123412 15299 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 13:11:01.123618 15299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 13:11:01.123641 15299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 13:11:01.123667 15299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 13:11:01.124795 15299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 13:11:01.124835 15299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 13:11:01.124853 15299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 13:11:01.124869 15299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 13:11:01.124890 15299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 13:11:01.126353 15299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.224.84:37277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.224.126:36291,127.13.224.125:34169,127.13.224.124:40201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/krb5kdc/kudu_127.13.224.124.keytab
--principal=kudu/127.13.224.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.224.124:40201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.224.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 11:44:25 UTC on ccycloud.quasar-gzggqh.root.comops.site
build id 4570062
I20260806 13:11:01.126590 15299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 13:11:01.126801 15299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 13:11:01.131593 15311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:11:01.131608 15309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 13:11:01.131606 15307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 13:11:01.131800 15299 server_base.cc:1034] running on GCE node
I20260806 13:11:01.132033 15299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 13:11:01.132304 15299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 13:11:01.133442 15299 hybrid_clock.cc:648] HybridClock initialized: now 1786021861133429 us; error 26 us; skew 500 ppm
Aug 06 13:11:01 dist-test-slave-q9h9 krb5kdc[15069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021861, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 13:11:01.135787 15299 init.cc:377] Logged in from keytab as kudu/127.13.224.124@KRBTEST.COM (short username kudu)
I20260806 13:11:01.136173 15299 webserver.cc:460] Webserver started at http://127.13.224.124:42083/ using document root <none> and password file <none>
I20260806 13:11:01.136373 15299 fs_manager.cc:359] Metadata directory not provided
I20260806 13:11:01.136420 15299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 13:11:01.136545 15299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 13:11:01.137567 15299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/data/instance:
uuid: "3eef534ac885474a87facd488c05f2f6"
format_stamp: "Formatted at 2026-08-06 13:11:01 on dist-test-slave-q9h9"
server_key: "f3fa75904b1ce6a13fabee9b3cc9b1b1"
server_key_iv: "039cab3e643727fc24d3bc612a7e438e"
server_key_version: "encryptionkey@0"
I20260806 13:11:01.137892 15299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/wal/instance:
uuid: "3eef534ac885474a87facd488c05f2f6"
format_stamp: "Formatted at 2026-08-06 13:11:01 on dist-test-slave-q9h9"
server_key: "f3fa75904b1ce6a13fabee9b3cc9b1b1"
server_key_iv: "039cab3e643727fc24d3bc612a7e438e"
server_key_version: "encryptionkey@0"
I20260806 13:11:01.139133 15299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 13:11:01.139808 15326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 13:11:01.139986 15299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 13:11:01.140067 15299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/data,/tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/wal
uuid: "3eef534ac885474a87facd488c05f2f6"
format_stamp: "Formatted at 2026-08-06 13:11:01 on dist-test-slave-q9h9"
server_key: "f3fa75904b1ce6a13fabee9b3cc9b1b1"
server_key_iv: "039cab3e643727fc24d3bc612a7e438e"
server_key_version: "encryptionkey@0"
I20260806 13:11:01.140126 15299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 13:11:01.157742 15299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 13:11:01.159752 15299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 13:11:01.159880 15299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 13:11:01.163537 15299 rpc_server.cc:307] RPC server started. Bound to: 127.13.224.124:40201
I20260806 13:11:01.163569 15400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.224.124:40201 every 8 connection(s)
I20260806 13:11:01.163869 15299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/data/info.pb
I20260806 13:11:01.164680 15401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 13:11:01.166112 15401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:11:01.167111 14209 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXc7v3G/build/release/bin/kudu as pid 15299
I20260806 13:11:01.167220 14209 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXc7v3G/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786021591455662-14209-0/minicluster-data/master-2/wal/instance
I20260806 13:11:01.167412 14209 external_mini_cluster.cc:1428] Setting key d9d05fba6136cc8b1581c4b116e39b9b
I20260806 13:11:01.168334 15185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } attempt: 3
Aug 06 13:11:01 dist-test-slave-q9h9 krb5kdc[15069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021861, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.124@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
Aug 06 13:11:01 dist-test-slave-q9h9 krb5kdc[15069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.124@KRBTEST.COM
Aug 06 13:11:01 dist-test-slave-q9h9 krb5kdc[15069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.126@KRBTEST.COM for kudu/127.13.224.124@KRBTEST.COM
I20260806 13:11:01.175005 15289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } attempt: 3
I20260806 13:11:01.176177 15190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.166724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59400 (local address 127.13.224.126:36291)
0806 13:11:01.166781 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.166782 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.166963 (+ 181us) server_negotiation.cc:407] Connection header received
0806 13:11:01.167597 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.167599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.167611 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.167670 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.168444 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.169047 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.169899 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.170117 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.170175 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.172496 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:01.172497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:01.172498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:01.172504 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:01.174541 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.175117 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.175118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.175119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.175175 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.175432 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.175433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.175434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.175618 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:01.175677 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:01.175988 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 13:11:01.176085 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 13:11:01.176285 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.166693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.126:36291 (local address 127.0.0.1:59400)
0806 13:11:01.166849 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:01.166887 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 13:11:01.167488 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:11:01.167758 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:11:01.167762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:11:01.168072 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:11:01.168347 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:01.168352 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.169109 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.169110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:01.169816 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:01.169819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.170184 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.170186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:01.170315 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.170717 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:11:01.170727 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:11:01.172412 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:11:01.174628 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:01.174635 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:01.174637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:01.175034 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:01.175239 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:01.175239 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:01.175240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:01.175360 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:01.175704 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:11:01.175710 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:11:01.175866 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 13:11:01.175963 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 13:11:01.176090 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":3,"thread_start_us":69,"threads_started":1}
I20260806 13:11:01.176856 15401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:11:01.177280 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.168582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40920)
0806 13:11:01.168631 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:01.168640 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:11:01.168674 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:11:01.168854 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:11:01.168854 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:11:01.169005 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:11:01.169044 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:01.169047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.170713 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.170714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:01.171453 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:01.171456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.171859 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.171860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:01.171919 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.172288 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:11:01.172293 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:11:01.173805 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:11:01.175909 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:01.175910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:01.175910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:01.176161 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:01.176613 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:01.176614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:01.176614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:01.176685 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:01.176999 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:11:01.177000 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:11:01.177044 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 13:11:01.177107 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 13:11:01.177194 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 13:11:01.177935 15408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.167787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40910 (local address 127.13.224.124:40201)
0806 13:11:01.168084 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.168088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.168106 (+ 18us) server_negotiation.cc:407] Connection header received
0806 13:11:01.168124 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.168126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.168147 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.168216 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.168969 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.169780 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.170847 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.171133 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.171198 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.173781 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:01.173790 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:01.173790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:01.173796 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:01.175738 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.176853 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.176855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.176855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.176934 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.177135 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.177136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.177136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.177260 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:01.177389 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:01.177723 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 13:11:01.177844 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":68,"threads_started":1}
I20260806 13:11:01.178399 15409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.168615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40920 (local address 127.13.224.124:40201)
0806 13:11:01.168735 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.168737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.168743 (+ 6us) server_negotiation.cc:407] Connection header received
0806 13:11:01.168761 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.168762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.168771 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.168836 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.169912 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.170633 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.171549 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.171788 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.171848 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.173913 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:01.173915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:01.173915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:01.173922 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:01.175844 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.176463 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.176467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.176467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.176531 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.176750 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.176751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.176751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.176895 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:01.177012 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:01.177635 (+ 623us) server_negotiation.cc:299] Negotiation successful
0806 13:11:01.177919 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
I20260806 13:11:01.179766 15185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7: Bootstrap starting.
Aug 06 13:11:01 dist-test-slave-q9h9 krb5kdc[15069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.125@KRBTEST.COM for kudu/127.13.224.124@KRBTEST.COM
I20260806 13:11:01.180385 15185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7: Neither blocks nor log segments found. Creating new log.
I20260806 13:11:01.180642 15185 log.cc:824] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7: Log is configured to *not* fsync() on all Append() calls
I20260806 13:11:01.181993 15185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7: No bootstrap required, opened a new log
Aug 06 13:11:01 dist-test-slave-q9h9 krb5kdc[15069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.126@KRBTEST.COM
I20260806 13:11:01.183784 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.175256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40924)
0806 13:11:01.175347 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:01.175356 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 13:11:01.175399 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:11:01.175688 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:11:01.175688 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:11:01.175819 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:11:01.175856 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:01.175858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.177373 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.177374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:01.178115 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:01.178119 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.178569 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.178570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:01.178626 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.178951 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:11:01.178955 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:11:01.180228 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:11:01.181933 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:01.181935 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:01.181935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:01.182151 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:01.183128 (+ 977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:01.183129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:01.183130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:01.183203 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:01.183473 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:11:01.183474 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:11:01.183520 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 13:11:01.183594 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 13:11:01.183691 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 13:11:01.183940 15185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } }
Aug 06 13:11:01 dist-test-slave-q9h9 krb5kdc[15069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021861, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.124@KRBTEST.COM for kudu/127.13.224.125@KRBTEST.COM
I20260806 13:11:01.184058 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.175351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40924 (local address 127.13.224.124:40201)
0806 13:11:01.175572 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.175574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.175579 (+ 5us) server_negotiation.cc:407] Connection header received
0806 13:11:01.175594 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.175594 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.175601 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.175662 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.175932 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.177290 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.178221 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.178481 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.178765 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.180299 (+ 1534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:01.180300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:01.180301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:01.180307 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:01.181841 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.182983 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.182984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.182984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.183044 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.183280 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.183281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.183281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.183397 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:01.183459 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:01.183897 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 13:11:01.183973 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":65,"threads_started":1}
I20260806 13:11:01.184073 15185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0145619affd84e1fa21f38a238637ac7, State: Initialized, Role: FOLLOWER
I20260806 13:11:01.184146 15185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } }
I20260806 13:11:01.184367 15416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } } }
I20260806 13:11:01.184435 15185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:11:01.184478 15416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:11:01.185758 15289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48: Bootstrap starting.
I20260806 13:11:01.186494 15289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48: Neither blocks nor log segments found. Creating new log.
I20260806 13:11:01.186793 15289 log.cc:824] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48: Log is configured to *not* fsync() on all Append() calls
I20260806 13:11:01.187232 15190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.178674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59402 (local address 127.13.224.126:36291)
0806 13:11:01.178702 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.178704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.178946 (+ 242us) server_negotiation.cc:407] Connection header received
0806 13:11:01.178965 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.178966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.178975 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.179028 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.179556 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.179976 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.181292 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.181562 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.181620 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.184056 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:01.184057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:01.184058 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:01.184163 (+ 105us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:01.185599 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.186143 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.186144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.186144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.186210 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.186413 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.186414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.186414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.186533 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:01.186666 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:01.187069 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 13:11:01.187139 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 13:11:01.187561 15289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48: No bootstrap required, opened a new log
I20260806 13:11:01.188233 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.177127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53346 (local address 127.13.224.125:34169)
0806 13:11:01.177163 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.177165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.177564 (+ 399us) server_negotiation.cc:407] Connection header received
0806 13:11:01.177603 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.177605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.177617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.177667 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.178110 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.178708 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.181647 (+ 2939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.182210 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.182284 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.185407 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:01.185408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:01.185408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:01.185415 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:01.187020 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.187411 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.187413 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.187413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.187470 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.187678 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.187679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.187679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.187846 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:01.187886 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:01.188076 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 13:11:01.188150 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 13:11:01.188495 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.177087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.125:34169 (local address 127.0.0.1:53346)
0806 13:11:01.177508 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:01.177516 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:11:01.177557 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:11:01.177667 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:11:01.177668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:11:01.177860 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:11:01.177907 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:01.177909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.181005 (+ 3096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.181006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:01.181571 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:01.181574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.182333 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.182334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:01.182400 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.182796 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:11:01.182800 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:11:01.185305 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:11:01.187140 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:01.187140 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:01.187141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:01.187328 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:01.187540 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:01.187540 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:01.187541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:01.187597 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:01.187930 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:11:01.187931 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:11:01.187969 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 13:11:01.188335 (+ 366us) client_negotiation.cc:240] Negotiation successful
0806 13:11:01.188416 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409}
I20260806 13:11:01.188768 15401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } has no permanent_uuid. Determining permanent_uuid...
I20260806 13:11:01.189433 15289 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } }
I20260806 13:11:01.189546 15289 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac7e82aae2924dc9b18c9dbbe0e5fd48, State: Initialized, Role: FOLLOWER
I20260806 13:11:01.187053 15088 master_runner.cc:186] Error getting master registration for 127.13.224.125:34169: OK, instance_id { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" instance_seqno: 1786021860927995 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 13:11:01.189610 15289 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } }
I20260806 13:11:01.189664 15088 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 13:11:01.189889 15289 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:11:01.190182 15432 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } } }
I20260806 13:11:01.190295 15432 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 13:11:01.191646 15450 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:11:01.191726 15450 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 06 13:11:01 dist-test-slave-q9h9 krb5kdc[15069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021861, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.224.124@KRBTEST.COM for kudu/127.13.224.124@KRBTEST.COM
Aug 06 13:11:01 dist-test-slave-q9h9 krb5kdc[15069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786021860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.224.125@KRBTEST.COM
I20260806 13:11:01.193233 15196 master_runner.cc:186] Error getting master registration for 127.13.224.124:40201: OK, instance_id { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" instance_seqno: 1786021861162525 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 13:11:01.195363 15196 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 13:11:01.195744 15429 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:11:01.195783 15429 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:11:01.197528 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.188945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.224.124:40201 (local address 127.0.0.1:40940)
0806 13:11:01.189051 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 13:11:01.189059 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 13:11:01.189088 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 13:11:01.189461 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 13:11:01.189461 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 13:11:01.189597 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0806 13:11:01.189621 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:01.189623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.190490 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.190490 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:01.191150 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 13:11:01.191153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.191640 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.191640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 13:11:01.191685 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.192004 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 13:11:01.192009 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 13:11:01.193828 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 13:11:01.195476 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:01.195477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:01.195478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:01.195686 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:01.195877 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 13:11:01.195878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 13:11:01.195878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 13:11:01.195927 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 13:11:01.197229 (+ 1302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 13:11:01.197230 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 13:11:01.197286 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 13:11:01.197359 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 13:11:01.197442 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":76,"threads_started":1}
I20260806 13:11:01.197654 15294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.187704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53360 (local address 127.13.224.125:34169)
0806 13:11:01.187735 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.187737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.187850 (+ 113us) server_negotiation.cc:407] Connection header received
0806 13:11:01.187867 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.187868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.187877 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.187917 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.188489 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.189195 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.190566 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.190891 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.190947 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.194001 (+ 3054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:01.194002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:01.194002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:01.194009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:01.195922 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.196433 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.196434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.196434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.196499 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.196815 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.196816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.196816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.196979 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:01.197133 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:01.197500 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 13:11:01.197577 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:11:01.197758 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.189328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40940 (local address 127.13.224.124:40201)
0806 13:11:01.189358 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.189359 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.189365 (+ 6us) server_negotiation.cc:407] Connection header received
0806 13:11:01.189381 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.189381 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.189388 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.189432 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.189687 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.190396 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.191243 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.191536 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.191577 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.193914 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:01.193915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:01.193915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:01.193921 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:01.195409 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.195767 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.195768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.195768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.195819 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.195979 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.195980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.195980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.196083 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:01.196135 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:01.197621 (+ 1486us) server_negotiation.cc:299] Negotiation successful
0806 13:11:01.197687 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 13:11:01.199496 15401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6: Bootstrap starting.
I20260806 13:11:01.200119 15401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6: Neither blocks nor log segments found. Creating new log.
I20260806 13:11:01.200441 15401 log.cc:824] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6: Log is configured to *not* fsync() on all Append() calls
I20260806 13:11:01.201303 14209 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 13:11:01.202200 15142 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0145619affd84e1fa21f38a238637ac7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59402
I20260806 13:11:01.202314 15142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 13:11:01.202355 15142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:11:01.202411 15401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6: No bootstrap required, opened a new log
I20260806 13:11:01.203172 15142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } }
I20260806 13:11:01.203394 15142 leader_election.cc:288] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [CANDIDATE]: Term 1 election: Requested vote from peers ac7e82aae2924dc9b18c9dbbe0e5fd48 (127.13.224.125:34169), 3eef534ac885474a87facd488c05f2f6 (127.13.224.124:40201)
I20260806 13:11:01.203687 15255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0145619affd84e1fa21f38a238637ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48"
I20260806 13:11:01.203884 15255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:11:01.204685 15255 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0145619affd84e1fa21f38a238637ac7 in term 1.
I20260806 13:11:01.204600 15401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } }
I20260806 13:11:01.204774 15401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eef534ac885474a87facd488c05f2f6, State: Initialized, Role: FOLLOWER
I20260806 13:11:01.204871 15108 leader_election.cc:302] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0145619affd84e1fa21f38a238637ac7, ac7e82aae2924dc9b18c9dbbe0e5fd48; no voters:
I20260806 13:11:01.204890 15401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } }
I20260806 13:11:01.204994 15416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 13:11:01.205149 15416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [term 1 LEADER]: Becoming Leader. State: Replica: 0145619affd84e1fa21f38a238637ac7, State: Running, Role: LEADER
I20260806 13:11:01.205164 15401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 13:11:01.205231 15416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } }
I20260806 13:11:01.205389 15363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0145619affd84e1fa21f38a238637ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3eef534ac885474a87facd488c05f2f6"
I20260806 13:11:01.205611 15416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0145619affd84e1fa21f38a238637ac7. Latest consensus state: current_term: 1 leader_uuid: "0145619affd84e1fa21f38a238637ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } } }
I20260806 13:11:01.205714 15416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [sys.catalog]: This master's current role is: LEADER
I20260806 13:11:01.205709 15363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 13:11:01.206523 15363 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0145619affd84e1fa21f38a238637ac7 in term 1.
I20260806 13:11:01.208691 15460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } } }
I20260806 13:11:01.208834 15460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:11:01.210214 15473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 13:11:01.210448 15473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 13:11:01.211697 15113 tablet.cc:2369] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 13:11:01.216418 15483 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 13:11:01.216475 15483 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 13:11:01.219102 15294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.210376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53372 (local address 127.13.224.125:34169)
0806 13:11:01.210409 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.210411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.211146 (+ 735us) server_negotiation.cc:407] Connection header received
0806 13:11:01.211225 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.211226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.211236 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.211291 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.212061 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.212768 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.213749 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.214083 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.214121 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.215824 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:01.215826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:01.215826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:01.215833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:01.217878 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.218278 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.218279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.218279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.218346 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.218538 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.218538 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.218539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.218651 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:01.218740 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:01.218944 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 13:11:01.219018 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:11:01.219219 15190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.210875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59410 (local address 127.13.224.126:36291)
0806 13:11:01.210907 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.210909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.211431 (+ 522us) server_negotiation.cc:407] Connection header received
0806 13:11:01.211517 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.211518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.211527 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.211553 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.212511 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.213200 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.214232 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.214533 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.214570 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.216579 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:01.216581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:01.216581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:01.216588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:01.218083 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.218467 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.218468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.218469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.218520 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.218705 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.218706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.218706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.218817 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:01.218880 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:01.219071 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 13:11:01.219142 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:11:01.222890 15255 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48 [term 1 FOLLOWER]: Refusing update from remote peer 0145619affd84e1fa21f38a238637ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:11:01.223138 15363 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6 [term 1 FOLLOWER]: Refusing update from remote peer 0145619affd84e1fa21f38a238637ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 13:11:01.223538 15461 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:11:01.223930 15416 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 13:11:01.225124 15460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0145619affd84e1fa21f38a238637ac7. Latest consensus state: current_term: 1 leader_uuid: "0145619affd84e1fa21f38a238637ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } } }
I20260806 13:11:01.225363 15432 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0145619affd84e1fa21f38a238637ac7. Latest consensus state: current_term: 1 leader_uuid: "0145619affd84e1fa21f38a238637ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } } }
I20260806 13:11:01.225440 15432 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:11:01.225297 15460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:11:01.227921 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:01.210664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40950 (local address 127.13.224.124:40201)
0806 13:11:01.210697 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.210698 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.211305 (+ 607us) server_negotiation.cc:407] Connection header received
0806 13:11:01.211359 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.211360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.211370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.211417 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.222094 (+ 10677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.222706 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.223644 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:01.223941 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:01.224001 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:01.225278 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 13:11:01.225279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 13:11:01.225279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 13:11:01.225286 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 13:11:01.226581 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.226948 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.226949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.226949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.227001 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 13:11:01.227173 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 13:11:01.227173 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 13:11:01.227174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 13:11:01.227285 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 13:11:01.227373 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:01.227767 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 13:11:01.227836 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 13:11:01.230640 15461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0145619affd84e1fa21f38a238637ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } } }
I20260806 13:11:01.230739 15461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [sys.catalog]: This master's current role is: LEADER
I20260806 13:11:01.230708 15460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0145619affd84e1fa21f38a238637ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } } }
I20260806 13:11:01.230768 15460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:11:01.230877 15487 mvcc.cc:205] Tried to move back new op lower bound from 7315545543566647296 to 7315545543497416704. Current Snapshot: MvccSnapshot[applied={T|T < 7315545543566647296}]
I20260806 13:11:01.231163 15492 mvcc.cc:205] Tried to move back new op lower bound from 7315545543566647296 to 7315545543497416704. Current Snapshot: MvccSnapshot[applied={T|T < 7315545543566647296}]
I20260806 13:11:01.234967 15461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0145619affd84e1fa21f38a238637ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } } }
I20260806 13:11:01.235046 15461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7 [sys.catalog]: This master's current role is: LEADER
I20260806 13:11:01.235332 15432 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0145619affd84e1fa21f38a238637ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0145619affd84e1fa21f38a238637ac7" member_type: VOTER last_known_addr { host: "127.13.224.126" port: 36291 } } peers { permanent_uuid: "ac7e82aae2924dc9b18c9dbbe0e5fd48" member_type: VOTER last_known_addr { host: "127.13.224.125" port: 34169 } } peers { permanent_uuid: "3eef534ac885474a87facd488c05f2f6" member_type: VOTER last_known_addr { host: "127.13.224.124" port: 40201 } } }
I20260806 13:11:01.235407 15432 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 13:11:01.238358 15494 mvcc.cc:205] Tried to move back new op lower bound from 7315545543566647296 to 7315545543497416704. Current Snapshot: MvccSnapshot[applied={T|T < 7315545543566647296}]
I20260806 13:11:01.238720 15473 catalog_manager.cc:1347] Generated new cluster ID: 30a3b515d6dc4bedaca2912899cf7378
I20260806 13:11:01.238765 15473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 13:11:01.266085 15473 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 13:11:01.266858 15473 catalog_manager.cc:1504] Loading token signing keys...
I20260806 13:11:01.294250 15473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0145619affd84e1fa21f38a238637ac7: Generated new TSK 0
I20260806 13:11:01.294440 15473 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 13:11:01.317482 15190 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:11:01.316464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59412 (local address 127.13.224.126:36291)
0806 13:11:01.316505 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.316507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.316809 (+ 302us) server_negotiation.cc:407] Connection header received
0806 13:11:01.316924 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.316925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.316936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.316965 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.317416 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59412: BlockingRecv error: recv got EOF from 127.0.0.1:59412 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 13:11:01.318660 15294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:11:01.317798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53386 (local address 127.13.224.125:34169)
0806 13:11:01.317833 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.317835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.318169 (+ 334us) server_negotiation.cc:407] Connection header received
0806 13:11:01.318260 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.318261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.318272 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.318300 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.318601 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53386: BlockingRecv error: recv got EOF from 127.0.0.1:53386 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 13:11:01.319408 15410 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:11:01.318042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40966 (local address 127.13.224.124:40201)
0806 13:11:01.318080 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.318081 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.318910 (+ 829us) server_negotiation.cc:407] Connection header received
0806 13:11:01.318994 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.318995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.319005 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.319034 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.319349 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40966: BlockingRecv error: recv got EOF from 127.0.0.1:40966 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 13:11:01.321462 15294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:11:01.320747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53402 (local address 127.13.224.125:34169)
0806 13:11:01.320781 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.320783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.320955 (+ 172us) server_negotiation.cc:407] Connection header received
0806 13:11:01.321051 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.321052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.321068 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.321096 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.321420 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53402: BlockingRecv error: recv got EOF from 127.0.0.1:53402 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 13:11:01.324331 15294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:11:01.323630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.13.224.125:34169)
0806 13:11:01.323662 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.323663 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.323868 (+ 205us) server_negotiation.cc:407] Connection header received
0806 13:11:01.323950 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.323951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.323959 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.323986 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.324290 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53410: BlockingRecv error: recv got EOF from 127.0.0.1:53410 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 13:11:01.327987 15294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:11:01.327333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53416 (local address 127.13.224.125:34169)
0806 13:11:01.327366 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.327368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.327546 (+ 178us) server_negotiation.cc:407] Connection header received
0806 13:11:01.327631 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.327632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.327641 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.327668 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.327949 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53416: BlockingRecv error: recv got EOF from 127.0.0.1:53416 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 13:11:01.333724 15294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:11:01.333014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53426 (local address 127.13.224.125:34169)
0806 13:11:01.333044 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.333045 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.333268 (+ 223us) server_negotiation.cc:407] Connection header received
0806 13:11:01.333358 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.333358 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.333368 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.333397 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.333684 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53426: BlockingRecv error: recv got EOF from 127.0.0.1:53426 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 13:11:01.343631 15294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:11:01.342889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.13.224.125:34169)
0806 13:11:01.342938 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.342940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.343168 (+ 228us) server_negotiation.cc:407] Connection header received
0806 13:11:01.343257 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.343258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.343267 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.343292 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.343592 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53428: BlockingRecv error: recv got EOF from 127.0.0.1:53428 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 13:11:01.361515 15294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:11:01.360812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53440 (local address 127.13.224.125:34169)
0806 13:11:01.360846 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.360848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.361053 (+ 205us) server_negotiation.cc:407] Connection header received
0806 13:11:01.361147 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.361148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.361158 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.361182 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.361473 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53440: BlockingRecv error: recv got EOF from 127.0.0.1:53440 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 13:11:01.395277 15294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:11:01.394650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53442 (local address 127.13.224.125:34169)
0806 13:11:01.394709 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.394711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.394813 (+ 102us) server_negotiation.cc:407] Connection header received
0806 13:11:01.394875 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.394876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.394886 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.394934 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.395223 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53442: BlockingRecv error: recv got EOF from 127.0.0.1:53442 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 13:11:01.460865 15294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:11:01.460277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.13.224.125:34169)
0806 13:11:01.460350 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.460352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.460413 (+ 61us) server_negotiation.cc:407] Connection header received
0806 13:11:01.460468 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.460469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.460479 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.460530 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.460806 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53452: BlockingRecv error: recv got EOF from 127.0.0.1:53452 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 13:11:01.590636 15294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:11:01.589909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53468 (local address 127.13.224.125:34169)
0806 13:11:01.589970 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.589973 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.590096 (+ 123us) server_negotiation.cc:407] Connection header received
0806 13:11:01.590169 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.590170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.590196 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.590243 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.590573 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53468: BlockingRecv error: recv got EOF from 127.0.0.1:53468 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 13:11:01.848347 15294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 13:11:01.847721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53478 (local address 127.13.224.125:34169)
0806 13:11:01.847776 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:01.847779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:01.847804 (+ 25us) server_negotiation.cc:407] Connection header received
0806 13:11:01.847884 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:01.847885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:01.847897 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:01.847942 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 13:11:01.848282 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53478: BlockingRecv error: recv got EOF from 127.0.0.1:53478 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 13:11:02.192157 15450 catalog_manager.cc:1259] Loaded cluster ID: 30a3b515d6dc4bedaca2912899cf7378
I20260806 13:11:02.192229 15450 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48: loading cluster ID for follower catalog manager: success
I20260806 13:11:02.193090 15450 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48: acquiring CA information for follower catalog manager: success
I20260806 13:11:02.193403 15450 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ac7e82aae2924dc9b18c9dbbe0e5fd48: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 13:11:02.216876 15483 catalog_manager.cc:1259] Loaded cluster ID: 30a3b515d6dc4bedaca2912899cf7378
I20260806 13:11:02.216934 15483 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6: loading cluster ID for follower catalog manager: success
I20260806 13:11:02.217703 15483 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6: acquiring CA information for follower catalog manager: success
I20260806 13:11:02.217984 15483 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3eef534ac885474a87facd488c05f2f6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 13:11:02.363572 15584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:02.361393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53482 (local address 127.13.224.125:34169)
0806 13:11:02.361497 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:02.361499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:02.361507 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:11:02.361533 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:02.361533 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:02.361580 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:02.361659 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:11:02.361791 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:02.362307 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:02.362988 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:02.363171 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:02.363213 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:02.363321 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:11:02.363424 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:11:02.363478 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:02.363490 (+ 12us) server_negotiation.cc:299] Negotiation successful
0806 13:11:02.363505 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":58,"threads_started":1}
I20260806 13:11:02.367367 15594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:02.365163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.13.224.124:40201)
0806 13:11:02.365272 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:02.365274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:02.365282 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:11:02.365323 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:02.365324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:02.365370 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:02.365434 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:11:02.365532 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:02.365997 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:02.366685 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:02.366889 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:02.366941 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:02.367071 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:11:02.367164 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:11:02.367236 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:02.367270 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 13:11:02.367289 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":52,"threads_started":1}
I20260806 13:11:02.371025 15584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:02.368719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53484 (local address 127.13.224.125:34169)
0806 13:11:02.368771 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:02.368773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:02.368869 (+ 96us) server_negotiation.cc:407] Connection header received
0806 13:11:02.368954 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:02.368956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:02.368964 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:02.369010 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:11:02.369141 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:02.369653 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:02.370486 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:02.370658 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:02.370712 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:02.370837 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:11:02.370862 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:11:02.370903 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:02.370945 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 13:11:02.370962 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 13:11:02.371642 15601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:02.368754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59428 (local address 127.13.224.126:36291)
0806 13:11:02.368992 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:02.368994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:02.369002 (+ 8us) server_negotiation.cc:407] Connection header received
0806 13:11:02.369046 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:02.369047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:02.369095 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:02.369165 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:11:02.369274 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:02.370043 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:02.370842 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:02.371116 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:02.371174 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:02.371332 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:11:02.371412 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:11:02.371463 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:02.371541 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 13:11:02.371564 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":47,"threads_started":1}
I20260806 13:11:02.372336 15594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 13:11:02.368959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40986 (local address 127.13.224.124:40201)
0806 13:11:02.368999 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 13:11:02.369000 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 13:11:02.369067 (+ 67us) server_negotiation.cc:407] Connection header received
0806 13:11:02.369119 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 13:11:02.369120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 13:11:02.369129 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 13:11:02.369174 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 13:11:02.369316 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:02.370035 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:02.371681 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 13:11:02.371949 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 13:11:02.372011 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 13:11:02.372123 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 13:11:02.372149 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 13:11:02.372205 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 13:11:02.372269 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 13:11:02.372285 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 13:11:02.373208 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 15088
I20260806 13:11:02.383313 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 15196
I20260806 13:11:02.392454 14209 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXc7v3G/build/release/bin/kudu with pid 15299
2026-08-06T13:11:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3363 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6232 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (270907 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST